* [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver
@ 2026-08-11 8:38 Ian Rogers
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
0 siblings, 3 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw)
To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel,
linux-perf-users, linux-rpi-kernel
Cc: Ian Rogers
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1948 ++++++++++++++++++++++++++++++++++++
3 files changed, 1959 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-11 8:38 ` Ian Rogers
2026-08-11 14:32 ` Uwe Kleine-König
2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw)
To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel,
linux-perf-users, linux-rpi-kernel
Cc: Ian Rogers
Add an uncore performance monitoring unit (PMU) driver for Broadcom
BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox
monitors found on Raspberry Pi 1 through 4.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1703 ++++++++++++++++++++++++++++++++++++
3 files changed, 1714 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..97c8bf34f463 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..adb34aefb26e
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1703 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/lockdep.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_axi_pmu_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id,
+ enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_axi_pmu_hrtimer_setup
+#endif
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (config_to_monitor(config) == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ int bus, filter, bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus = pmu->monitor[mon].hw_events.monitored_bus[idx];
+ filter = pmu->monitor[mon].hw_events.filter[idx];
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = true;
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ ret = tmp[2];
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx]) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = new_count - prev_count;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx);
+ spin_lock_irq(&pmu->lock);
+ }
+
+ /* Drop spinlock during Mailbox IPC read (which can sleep) */
+ spin_unlock_irq(&pmu->lock);
+
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+
+ spin_lock_irq(&pmu->lock);
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = new_count - prev_count;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = new_count - prev_count;
+ local64_add(delta, &event->count);
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx);
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, HRTIMER_MODE_REL_PINNED);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE) {
+ rpi_axi_pmu_read(event);
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ if (mon == MON__SYSTEM)
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0)
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = CHIP_BCM2835;
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = raw_smp_processor_id();
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ if (!pmu->firmware)
+ return -EPROBE_DEFER;
+ }
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ ret = of_property_read_u32_index(dev->of_node, "reg", i * 2,
+ &pmu->monitor[i].mailbox);
+ if (ret) {
+ dev_err(dev, "Error reading mailbox reg %d %d\n", i * 2, ret);
+ goto err_firmware_put;
+ }
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_firmware_put;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp;
+ }
+ return 0;
+
+err_cpuhp:
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ perf_pmu_unregister(&pmu->pmu);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {},
+};
+
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-11 8:38 ` Ian Rogers
2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 0 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-11 8:38 UTC (permalink / raw)
To: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel,
linux-perf-users, linux-rpi-kernel
Cc: Ian Rogers
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 4 +-
drivers/perf/rpi_axi_pmu.c | 259 ++++++++++++++++++++++++++++++++++++-
2 files changed, 254 insertions(+), 9 deletions(-)
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 97c8bf34f463..b0c4eb0748dd 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU
config RASPBERRYPI_AXI_PMU
tristate "Raspberry Pi AXI PMU support"
- depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST
select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
help
Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
- SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
If compiled as a module, it will be called rpi_axi_pmu.
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index adb34aefb26e..ec1327804c00 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,12 +6,13 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
* The Broadcom AXI performance hardware provides up to two independent monitors:
* 1. System Monitor (MON__SYSTEM = 0):
- * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
* Directly memory-mapped via ARM physical IO memory space (MMIO).
* Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
*
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -75,9 +77,11 @@ static inline void rpi_axi_pmu_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -129,6 +133,44 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
* @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
@@ -261,6 +303,94 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
#define GEN_CTL_ENABLE_BIT BIT(0)
@@ -483,16 +613,22 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (config >> 15 != 0)
return false;
- if (config_to_monitor(config) == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (config_to_monitor(config) == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
-
return counter < CNT__MAX;
}
@@ -719,6 +855,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -875,12 +1049,79 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (pmu->chip == CHIP_BCM2712) {
+ /* On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1) */
+ if (strstr(pmu_attr->event_str, "monitor=1"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "rp1_pcie"))
+ return 0;
+ }
+
+ return attr->mode;
+}
+
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -1648,6 +1889,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{},
};
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-11 14:32 ` Uwe Kleine-König
0 siblings, 0 replies; 59+ messages in thread
From: Uwe Kleine-König @ 2026-08-11 14:32 UTC (permalink / raw)
To: Ian Rogers
Cc: Will Deacon, Mark Rutland, linux-kernel, linux-arm-kernel,
linux-perf-users, linux-rpi-kernel
[-- Attachment #1: Type: text/plain, Size: 967 bytes --]
Hello,
On Tue, Aug 11, 2026 at 01:38:26AM -0700, Ian Rogers wrote:
> +#include <linux/mod_devicetable.h>
> [...]
> +#include <linux/platform_device.h>
Please rely on <linux/platform_device.h> to provide the definition of
of_device_id and drop <linux/mod_devicetable.h>. The latter will go away
soon.
> +/* Devices matching this driver in Device Tree */
> +static const struct of_device_id rpi_axi_pmu_match[] = {
> + {
> + .compatible = "brcm,bcm2835-axiperf",
> + .data = (void *)CHIP_BCM2835,
> + },
> + {
> + .compatible = "brcm,bcm2711-axiperf",
> + .data = (void *)CHIP_BCM2835,
> + },
> + {},
Please use
{ }
as list terminator (that is, add a space and drop the comma).
Also if the data is the same for all variants, maybe better drop the
assigment to .data and hardcode the value accordingly.
> +};
> +
> +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
Usually there is no empty line between the device_id struct and the
MODULE_DEVICE_TABLE macro.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-11 23:51 ` Ian Rogers
2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
2 siblings, 3 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, will,
Uwe Kleine-König
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Changes in v2
-------------
- Match Data Initialization: In rpi_axi_pmu__init(), set pmu->chip via
of_device_get_match_data(dev) to ensure BCM2712 match data is correctly parsed.
- Sysfs Event Visibility Filtering: Updated rpi_axi_pmu_events_is_visible() to
properly filter all 7 BCM2712 master filtered event aliases (arm_, pcie0_,
hvs_filtered, emmc0_) on BCM2835-BCM2711 systems, and hide legacy RPi 1-4
aliases on BCM2712 systems.
- Unmapped Monitor Check: Added monitor base_address and initialization check to
config_is_valid(). Events targeting uninitialized or unmapped monitors (e.g. missing
memory resource or monitor=1 on BCM2712) are properly rejected with -EINVAL.
- CPU Affinity Enforcement: Restrict uncore events in event_init() to event->cpu == pmu->cpu
to prevent orphaned uncore events when a non-PMU CPU goes offline.
- 31-bit Counter Wrap Delta: Mask counter accumulation with delta = (new_count - prev_count) & 0x7FFFFFFF
in read(), timer_handler(), and vpu_work_handler() to prevent 2B+ event spikes on counter overflow.
- Stopped Event Guard: Return early in rpi_axi_pmu_read() if event->hw.state & PERF_HES_STOPPED.
- Lockless Race Fix: Pass bus and filter explicitly to rpi_axi_pmu_enable_bus_watcher()
under spinlock to prevent lockless read races in vpu_work_handler().
- Slot Reuse Clean: Set enabled[idx] = false in rpi_axi_pmu_del() when refcount[idx] == 0 so
re-allocated watcher slots properly enable hardware watchers.
- Events Array Bounds: Return -ENOSPC in rpi_axi_pmu_add() if pmu->events array is full.
- Timer Cancel Race Fix: Re-acquire pmu->lock after hrtimer_cancel() in rpi_axi_pmu_del()
and re-verify active_events > 0 before returning.
- Hotplug Timer Migration: Cancel and re-pin pmu->hrtimer to target CPU in rpi_axi_pmu_offline_cpu().
- Registration Order: Call perf_pmu_register() before cpuhp_state_add_instance() in probe,
and cancel hrtimer before cancel_work_sync() and unregister in exit teardown.
- Header & Style Cleanups: Removed <linux/mod_devicetable.h> and <linux/version.h>,
updated rpi_axi_pmu_match[] terminator to { }, and fixed C variable declaration style.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2001 ++++++++++++++++++++++++++++++++++++
3 files changed, 2012 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-11 23:51 ` Ian Rogers
2026-08-12 0:04 ` sashiko-bot
2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, will,
Uwe Kleine-König
Add an uncore performance monitoring unit (PMU) driver for Broadcom
BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox
monitors found on Raspberry Pi 1 through 4.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1739 ++++++++++++++++++++++++++++++++++++
3 files changed, 1750 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..97c8bf34f463 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..cdf7d9a74f13
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1739 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = true;
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ ret = tmp[2];
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ }
+
+ /* Drop spinlock during Mailbox IPC read (which can sleep) */
+ spin_unlock_irq(&pmu->lock);
+
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+
+ spin_lock_irq(&pmu->lock);
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, HRTIMER_MODE_REL_PINNED);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE) {
+ rpi_axi_pmu_read(event);
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ if (mon == MON__SYSTEM)
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0)
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_PINNED);
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_PINNED);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = CHIP_BCM2835;
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = raw_smp_processor_id();
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ if (!pmu->firmware)
+ return -EPROBE_DEFER;
+ }
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ ret = of_property_read_u32_index(dev->of_node, "reg", i * 2,
+ &pmu->monitor[i].mailbox);
+ if (ret) {
+ dev_err(dev, "Error reading mailbox reg %d %d\n", i * 2, ret);
+ goto err_firmware_put;
+ }
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_firmware_put;
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_pmu_unregister;
+ }
+ return 0;
+
+err_pmu_unregister:
+ perf_pmu_unregister(&pmu->pmu);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ perf_pmu_unregister(&pmu->pmu);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-11 23:51 ` Ian Rogers
2026-08-12 0:05 ` sashiko-bot
2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-11 23:51 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, will,
Uwe Kleine-König
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 4 +-
drivers/perf/rpi_axi_pmu.c | 278 +++++++++++++++++++++++++++++++++++--
2 files changed, 272 insertions(+), 10 deletions(-)
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 97c8bf34f463..b0c4eb0748dd 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU
config RASPBERRYPI_AXI_PMU
tristate "Raspberry Pi AXI PMU support"
- depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST
select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
help
Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
- SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
If compiled as a module, it will be called rpi_axi_pmu.
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index cdf7d9a74f13..b13632fa8f84 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,12 +6,13 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
* The Broadcom AXI performance hardware provides up to two independent monitors:
* 1. System Monitor (MON__SYSTEM = 0):
- * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
* Directly memory-mapped via ARM physical IO memory space (MMIO).
* Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
*
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -60,9 +62,11 @@
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -114,6 +118,44 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
* @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
@@ -246,6 +288,94 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
#define GEN_CTL_ENABLE_BIT BIT(0)
@@ -475,16 +605,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
-
return counter < CNT__MAX;
}
@@ -711,6 +849,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -867,12 +1043,94 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and RPi 1-4 aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_"))
+ return 0;
+ }
+
+ return attr->mode;
+}
+
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -1549,7 +1807,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
spin_lock_init(&pmu->lock);
mutex_init(&pmu->vpu_mutex);
- pmu->chip = CHIP_BCM2835;
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
pmu->pmu = (struct pmu) {
.module = THIS_MODULE,
@@ -1685,6 +1943,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 0:04 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 0:04 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider:
- [High] The probe function ignores a missing 'firmware' device tree phandle, leaving `pmu->firmware` NULL but leaving the VPU monitor enabled, causing a later NULL pointer dereference in the mailbox IPC.
- [High] The driver fails to prevent sysfs unbinding (`.suppress_bind_attrs = true`), leading to a Use-After-Free if unbound while perf events are active.
- [High] Incorrect manual parsing of the 'reg' property assumes fixed address and size cells, breaking on 64-bit platforms.
- [High] Incorrect PMU teardown ordering in `rpi_axi_pmu__exit` and missing cleanup in probe error paths cause Use-After-Free vulnerabilities.
- [High] The PMU driver uses a `spinlock_t` in a hardirq hrtimer callback, causing lockdep splats and sleeping in atomic context on PREEMPT_RT.
- [Medium] An unprotected write to `hw_events.enabled[idx]` creates a race condition with event deletion, which can silently break hardware configuration for recycled bus watcher slots.
- [Medium] The CPU hotplug offline callback erroneously restarts the polling hrtimer pinned to the dying CPU, sabotaging PMU migration.
- [Low] Raspberry Pi 5 (BCM2712) support is partially implemented but completely unreachable because the chip type is hardcoded to BCM2835 and the device tree match table lacks BCM2712.
--
--- Patch [1]: [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit 3567c05975da01f684191bbbfa826e0e8d8e9927
Author: Ian Rogers <irogers@google.com>
Subject: perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit introduces a new uncore PMU driver for Broadcom BCM2835-BCM2711
AXI system monitors. It supports tracking AXI traffic via MMIO for the system
monitor and via mailbox IPC for the VPU monitor.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..cdf7d9a74f13e
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
> +{
[ ... ]
> + pmu->chip = CHIP_BCM2835;
[Severity: Low]
Is BCM2712 hardware meant to be supported here?
The chip type is unconditionally hardcoded to CHIP_BCM2835, which appears to
make the BCM2712 definitions unreachable.
> +
> + pmu->pmu = (struct pmu) {
[ ... ]
> + if (pmu->chip == CHIP_BCM2835) {
> + pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
> + pmu->monitor[MON__VPU].use_mailbox_interface = true;
> + fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
> + if (fw_node) {
> + pmu->firmware = rpi_firmware_get(fw_node);
> + of_node_put(fw_node);
> + if (!pmu->firmware)
> + return -EPROBE_DEFER;
> + }
[Severity: High]
Can this lead to a NULL pointer dereference in the mailbox IPC?
If the device tree lacks the firmware property, fw_node is NULL and
the driver continues with pmu->firmware left as NULL.
Later, set_monitor_control() will call rpi_firmware_property() using this
NULL pointer when VPU events are started.
> + } else {
[ ... ]
> + for (int i = 0; i < MON__MAX; i++) {
> + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
> +
> + if (pmu->monitor[i].use_mailbox_interface) {
> + ret = of_property_read_u32_index(dev->of_node, "reg", i * 2,
> + &pmu->monitor[i].mailbox);
[Severity: High]
Does this correctly parse the reg property on 64-bit platforms?
Using of_property_read_u32_index() with a fixed index of i * 2 assumes that
address and size cells are exactly 1.
On platforms with 64-bit device tree layouts, could this fetch the wrong
word and corrupt the mailbox address?
> + if (ret) {
[ ... ]
> + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
> + if (ret) {
> + dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
> + goto err_pmu_unregister;
> + }
> + return 0;
> +
> +err_pmu_unregister:
> + perf_pmu_unregister(&pmu->pmu);
> +err_firmware_put:
> + if (pmu->firmware)
> + rpi_firmware_put(pmu->firmware);
> + return ret;
> +}
[Severity: High]
Could this regression cause a use-after-free if probe fails here?
If cpuhp_state_add_instance() fails, the error path unregisters the PMU but
does not cancel the hrtimer or vpu_work.
Since the PMU is briefly visible in sysfs, might userspace race to open an
event and arm the timer before the context is freed by devres?
> +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
> +{
> + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
> + hrtimer_cancel(&pmu->hrtimer);
> + cancel_work_sync(&pmu->vpu_work);
> + if (pmu->firmware)
> + rpi_firmware_put(pmu->firmware);
> + perf_pmu_unregister(&pmu->pmu);
> +}
[Severity: High]
Does this teardown ordering allow new events to race with driver removal?
By cancelling the timer and workqueue before unregistering the PMU, there
appears to be a window where new events can be scheduled that restart the
timers while the context is being torn down.
[ ... ]
> +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
> + int idx, int bus, int filter)
> +{
[ ... ]
> + pmu->monitor[mon].hw_events.enabled[idx] = true;
> +}
[Severity: Medium]
Is this state update safe from concurrent deletion?
The array element is modified without holding pmu->lock.
When rpi_axi_pmu_vpu_work_handler() drops the lock to call this function:
rpi_axi_pmu_vpu_work_handler()
spin_unlock_irq(&pmu->lock);
rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
spin_lock_irq(&pmu->lock);
A concurrent rpi_axi_pmu_del() could clear the slot and set enabled to false.
If a new event immediately reuses this slot, wouldn't the unlocked write to
true overwrite the hardware configuration state for the new event?
[ ... ]
> +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
> +{
> + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
> + unsigned long flags;
> +
> + spin_lock_irqsave(&pmu->lock, flags);
[Severity: High]
Will this lock acquisition panic on PREEMPT_RT kernels?
The timer defaults to hardirq context, where acquiring a sleepable lock like
spinlock_t triggers a panic on PREEMPT_RT.
[ ... ]
> +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
> +{
[ ... ]
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + if (pmu->active_events > 0) {
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> + hrtimer_cancel(&pmu->hrtimer);
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + if (pmu->active_events > 0)
> + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
> + HRTIMER_MODE_REL_PINNED);
> + }
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> +
> + return 0;
> +}
[Severity: Medium]
Does this logic unintentionally sabotage PMU migration during CPU offline?
This callback runs in the context of the dying CPU.
By passing HRTIMER_MODE_REL_PINNED here, the timer seems to be pinned to the
offlining CPU instead of being successfully migrated to the target CPU.
[ ... ]
> +static int rpi_axi_pmu_probe(struct platform_device *pdev)
> +{
> + struct rpi_axi_pmu *pmu;
> +
> + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
[ ... ]
> +static const struct of_device_id rpi_axi_pmu_match[] = {
> + {
> + .compatible = "brcm,bcm2835-axiperf",
> + .data = (void *)CHIP_BCM2835,
> + },
> + {
> + .compatible = "brcm,bcm2711-axiperf",
> + .data = (void *)CHIP_BCM2835,
> + },
> + { }
> +};
[Severity: Low]
Could this device tree match table be missing a compatible string for
BCM2712 hardware, preventing it from probing entirely?
> +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
> +
> +static struct platform_driver rpi_axi_pmu_driver = {
> + .probe = rpi_axi_pmu_probe,
> + .remove = rpi_axi_pmu_remove,
> + .driver = {
> + .name = PMU_NAME,
> + .of_match_table = of_match_ptr(rpi_axi_pmu_match),
> + },
> +};
[Severity: High]
Can an administrator unbinding this driver via sysfs trigger a use-after-free?
The pmu context is allocated with devm_kzalloc(), tying its memory lifetime
to the device bind state.
Since .suppress_bind_attrs = true is omitted, a sysfs unbind will free the
memory while active perf events might still exist and try to access the
freed context.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260811235112.2267241-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 0:05 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 0:05 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Incorrect subsystem teardown ordering in both the exit and init error paths fails to cancel background tasks, leading to Use-After-Free.
- [Medium] Incomplete hiding of legacy BCM2835-specific sysfs event aliases on BCM2712.
- [Medium] Dropping the spinlock in the VPU work handler causes a TOCTOU race that corrupts hardware configurations.
- [Low] Invalid Kconfig dependency on non-existent symbol `ARCH_BCM2712`.
--
--- Patch [2]: [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support ---
commit c6c37f9c3fb56c6bdbc0982c7918061dbe1d45cb
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
> diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
> index 97c8bf34f4633..b0c4eb0748dd3 100644
> --- a/drivers/perf/Kconfig
> +++ b/drivers/perf/Kconfig
> @@ -327,11 +327,11 @@ config NVIDIA_TEGRA410_C2C_PMU
>
> config RASPBERRYPI_AXI_PMU
> tristate "Raspberry Pi AXI PMU support"
> - depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
> + depends on ARCH_BCM2835 || ARCH_BCM2712 || ARCH_BRCMSTB || COMPILE_TEST
[Severity: Low]
Does this code add a dependency on an undefined symbol?
ARCH_BCM2712 does not appear to be defined anywhere in the kernel tree.
> select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
> help
> Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
> - SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
> + SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
>
> If compiled as a module, it will be called rpi_axi_pmu.
>
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> index cdf7d9a74f13e..b13632fa8f84c 100644
> --- a/drivers/perf/rpi_axi_pmu.c
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> @@ -867,12 +1043,94 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
[ ... ]
> +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
> + struct attribute *attr, int unused)
> +{
> + struct device *dev = kobj_to_dev(kobj);
> + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
> + struct perf_pmu_events_attr *pmu_attr;
> +
> + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
> +
> + if (pmu->chip == CHIP_BCM2712) {
> + /*
> + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
> + * and RPi 1-4 aliases
> + */
> + if (strstr(pmu_attr->event_str, "monitor=1") ||
> + strstr(attr->name, "cpu0_") ||
> + strstr(attr->name, "cpu1_") ||
> + strstr(attr->name, "dma0_") ||
> + strstr(attr->name, "dma1_") ||
> + strstr(attr->name, "v3d0_") ||
> + strstr(attr->name, "v3d1_") ||
> + strstr(attr->name, "hvs_system") ||
> + strstr(attr->name, "isp_system") ||
> + strstr(attr->name, "usb_system"))
> + return 0;
[Severity: Medium]
Are there missing legacy aliases that should be hidden here?
This hides VPU IPC events on RPi 5, but it looks like several legacy System
Monitor aliases for diverged buses (such as ccp2tx, mphi_rx, h264, v3d,
peripheral, cpu_uc, cpu_l2) are not included in this check.
Opening them might measure the wrong hardware unit or fail with -EINVAL since
their hardcoded bus indices exceed BCM2712_SB__MAX.
> + } else {
> + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
> + if (strstr(attr->name, "pcie_rp1") ||
> + strstr(attr->name, "a76_") ||
> + strstr(attr->name, "v3d7_") ||
> + strstr(attr->name, "hevc_dec") ||
> + strstr(attr->name, "arm_") ||
> + strstr(attr->name, "pcie0_") ||
> + strstr(attr->name, "hvs_filtered") ||
> + strstr(attr->name, "emmc0_"))
> + return 0;
> + }
> +
> + return attr->mode;
> +}
[ ... ]
> @@ -1549,7 +1807,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
> spin_lock_init(&pmu->lock);
> mutex_init(&pmu->vpu_mutex);
>
> - pmu->chip = CHIP_BCM2835;
> + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
>
> pmu->pmu = (struct pmu) {
> .module = THIS_MODULE,
[Severity: High]
Can the error paths in rpi_axi_pmu__init() and the teardown in
rpi_axi_pmu__exit() lead to a use-after-free?
If cpuhp_state_add_instance() fails in rpi_axi_pmu__init(), the driver calls
perf_pmu_unregister() but doesn't cancel the background tasks:
rpi_axi_pmu__init() {
...
err_pmu_unregister:
perf_pmu_unregister(&pmu->pmu);
err_firmware_put:
if (pmu->firmware)
rpi_firmware_put(pmu->firmware);
return ret;
}
Since PMU memory is freed via devres upon returning the error, an active timer
or workqueue (started by an event opened during the registration window) could
fire and access freed memory.
Similarly, in rpi_axi_pmu__exit(), the background tasks are cancelled before
unregistering the PMU subsystem:
rpi_axi_pmu__exit() {
cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
hrtimer_cancel(&pmu->hrtimer);
cancel_work_sync(&pmu->vpu_work);
...
perf_pmu_unregister(&pmu->pmu);
}
A concurrent userspace action could start a new event in this window,
restarting the timer and causing a use-after-free when the memory is
subsequently freed.
[Severity: Medium]
Is there a time-of-check to time-of-use race condition in
rpi_axi_pmu_vpu_work_handler()?
When the lock is dropped, a concurrent event deletion and addition could
potentially misconfigure the PMU watcher:
rpi_axi_pmu_vpu_work_handler() {
...
if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
spin_unlock_irq(&pmu->lock);
rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
spin_lock_irq(&pmu->lock);
}
...
}
Concurrently, rpi_axi_pmu_del() can delete the event and free the idx slot,
which rpi_axi_pmu_add() can then reuse for a new event with different
configurations.
Because the old values were cached before dropping the lock, they might be
forcefully written to the hardware, and enabled[idx] is modified outside the
lock:
rpi_axi_pmu_enable_bus_watcher() {
...
pmu->monitor[mon].hw_events.enabled[idx] = true;
...
}
Could this silently misconfigure the PMU watcher for the new event?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260811235112.2267241-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 0:27 ` Ian Rogers
2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
2 siblings, 3 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context,
ensuring spinlock acquisitions are 100% PREEMPT_RT safe.
- Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while
active perf events exist.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Changes in v3
-------------
- 64-bit Device Tree reg Parsing: Replaced of_property_read_u32_index with
platform_get_resource(pdev, IORESOURCE_MEM, i) to retrieve mailbox offsets,
supporting 64-bit DT layouts (#address-cells = <2>).
- Firmware Phandle Fallback Safety: In rpi_axi_pmu__init(), if firmware node is
missing or rpi_firmware_get() returns NULL, explicitly set MON__VPU use_mailbox_interface = false.
config_is_valid() then returns -EINVAL for VPU events, preventing NULL pointer dereferences.
- Driver Teardown Order: In rpi_axi_pmu__exit(), unregister perf_pmu_unregister(&pmu->pmu)
first to block new event allocations before removing CPU hotplug callbacks, cancelling hrtimer,
and syncing workqueue.
- Probe Error Unwind Safety: Updated rpi_axi_pmu__init() error paths to call hrtimer_cancel()
and cancel_work_sync() before returning probe failure.
- Thread-Safe enabled[idx] Setting: Set enabled[idx] = true under pmu->lock after verifying
event generation sequence in vpu_work_handler(), eliminating TOCTOU race conditions.
- PREEMPT_RT Compatibility: Switched hrtimer setup and start to HRTIMER_MODE_REL_SOFT
so timer callbacks run in softirq context, allowing spinlock_t acquisitions on PREEMPT_RT.
- CPU Offline Timer Migration: In rpi_axi_pmu_offline_cpu(), restart hrtimer using unpinned
HRTIMER_MODE_REL_SOFT so perf migrates timer execution to the target CPU.
- Suppress Unsafe Sysfs Unbind: Set .suppress_bind_attrs = true in platform_driver.
- Kconfig Cleanup: Removed non-existent ARCH_BCM2712 symbol from depends on in Kconfig.
- Legacy Sysfs Alias Filtering: Updated rpi_axi_pmu_events_is_visible() to filter out all
legacy RPi 1-4 System Monitor aliases (ccp2tx_, mphi_, h264_, v3d_, jpeg_, peripheral_,
cpu_uc_, cpu_l2_, etc.) on BCM2712 systems.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2022 ++++++++++++++++++++++++++++++++++++
3 files changed, 2033 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-12 0:27 ` Ian Rogers
2026-08-12 0:40 ` sashiko-bot
2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Add an uncore performance monitoring unit (PMU) driver for Broadcom
BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox
monitors found on Raspberry Pi 1 through 4.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1748 ++++++++++++++++++++++++++++++++++++
3 files changed, 1759 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..97c8bf34f463 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..1923dc34cb3e
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1748 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ ret = tmp[2];
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ }
+
+ /* Drop spinlock during Mailbox IPC read (which can sleep) */
+ spin_unlock_irq(&pmu->lock);
+
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+
+ spin_lock_irq(&pmu->lock);
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE) {
+ rpi_axi_pmu_read(event);
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ if (mon == MON__SYSTEM)
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0)
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = CHIP_BCM2835;
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = raw_smp_processor_id();
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_pmu_unregister;
+ }
+ return 0;
+
+err_pmu_unregister:
+ perf_pmu_unregister(&pmu->pmu);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ perf_pmu_unregister(&pmu->pmu);
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 0:27 ` Ian Rogers
2026-08-12 0:40 ` sashiko-bot
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 0:27 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 2 +-
drivers/perf/rpi_axi_pmu.c | 290 ++++++++++++++++++++++++++++++++++++-
2 files changed, 283 insertions(+), 9 deletions(-)
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 97c8bf34f463..84ffa65c7b79 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU
select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
help
Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
- SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
If compiled as a module, it will be called rpi_axi_pmu.
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index 1923dc34cb3e..478d58c7e4f4 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,12 +6,13 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
* The Broadcom AXI performance hardware provides up to two independent monitors:
* 1. System Monitor (MON__SYSTEM = 0):
- * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
* Directly memory-mapped via ARM physical IO memory space (MMIO).
* Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
*
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -60,9 +62,11 @@
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -114,6 +118,44 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
* @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
@@ -246,6 +288,94 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
#define GEN_CTL_ENABLE_BIT BIT(0)
@@ -475,16 +605,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
-
return counter < CNT__MAX;
}
@@ -711,6 +849,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -867,12 +1043,106 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ strstr(attr->name, "jpeg_") ||
+ strstr(attr->name, "peripheral_") ||
+ strstr(attr->name, "cpu_uc_") ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_"))
+ return 0;
+ }
+
+ return attr->mode;
+}
+
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -1550,7 +1820,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
spin_lock_init(&pmu->lock);
mutex_init(&pmu->vpu_mutex);
- pmu->chip = CHIP_BCM2835;
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
pmu->pmu = (struct pmu) {
.module = THIS_MODULE,
@@ -1693,6 +1963,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 0:40 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 0:40 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] Calling hrtimer_cancel() inside rpi_axi_pmu_del() causes an unrecoverable deadlock if the PMU event is deleted via a cross-CPU IPI while the timer callback is running.
- [High] Global monitor reset is asserted every time a single bus watcher is enabled, corrupting other active events.
- [High] When MON__VPU falls back to MMIO, the vpu_work_handler drops the spinlock before accessing hardware, triggering lockdep assertions and unprotected MMIO writes.
- [Medium] rpi_axi_pmu_read_counter() returns -1 (0xFFFFFFFF) when a mailbox IPC failure occurs, causing massive artificial spikes in the reported event counts.
- [Medium] Race condition during probe allows the PMU to be permanently bound to an offline CPU.
--
--- Patch [1]: [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit b18036baa6b636e4ea8f9af5eee5462724c801ef
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds an uncore performance monitoring unit (PMU) driver for
Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..1923dc34cb3e9
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
> + int idx, int bus, int filter)
> +{
> + int bus_control;
> +
> + if (pmu->monitor[mon].hw_events.enabled[idx])
> + return;
> +
> + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
> + if (filter) {
> + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
> + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
> + }
> + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
> + set_bus_watcher_control(pmu, mon, idx, bus_control);
> +
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
[Severity: High]
Does this global reset corrupt other active events?
It appears that whenever a new event is started, this unconditionally
issues the reset bit to the global monitor control register. This might
destroy the hardware state and clear the counters of all other currently
running bus watchers, resulting in massive phantom spikes on their next
read due to underflow.
> + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
> +}
[ ... ]
> +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
> + enum counter counter)
> +{
> + int watcher = watcher_offset(pmu, mon, idx);
> + int offset = counter_offset(counter);
> + u32 ret;
> +
> + if (!pmu->monitor[mon].hw_events.enabled[idx])
> + return 0;
> +
> + if (pmu->monitor[mon].use_mailbox_interface) {
> + u32 tmp[3] = {
> + pmu->monitor[mon].mailbox + watcher + offset,
> + 1, -1
> + };
> + int err;
> +
> + might_sleep();
> + lockdep_assert_held(&pmu->vpu_mutex);
> + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
> + return -1;
> +
> + err = rpi_firmware_property(pmu->firmware,
> + RPI_FIRMWARE_GET_PERIPH_REG,
> + tmp, sizeof(tmp));
> +
> + if (err < 0 || tmp[1] != 1)
> + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
> + ret = tmp[2];
[Severity: Medium]
Is it safe to return the uninitialized buffer on an IPC failure?
If the firmware mailbox call fails or times out, it looks like the response
buffer tmp[2] remains -1, and that is returned directly. This could be
treated as a huge jump (0xFFFFFFFF) in the delta calculation and result in
massive artificial spikes in the reported event counts.
> + } else {
> + void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
> +
> + lockdep_assert_held(&pmu->lock);
> + ret = readl(addr) & 0x7FFFFFFF;
> + }
> + return ret;
> +}
[ ... ]
> +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
> +{
> + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
> +
> + might_sleep();
> + mutex_lock(&pmu->vpu_mutex);
> + spin_lock_irq(&pmu->lock);
> +
> + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
[ ... ]
> + /* Drop spinlock during Mailbox IPC read (which can sleep) */
> + spin_unlock_irq(&pmu->lock);
> +
> + new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
> +
> + spin_lock_irq(&pmu->lock);
[Severity: High]
Could this drop of the spinlock trigger a lockdep warning and data race
if the monitor falls back to MMIO?
If the firmware is unavailable, the VPU monitor might fall back to direct
MMIO (use_mailbox_interface = false). The work handler assumes it uses
IPC and drops the lock. But the MMIO path in rpi_axi_pmu_read_counter()
asserts lockdep_assert_held(&pmu->lock), and performs raw readl operations,
potentially racing with synchronous MMIO reads in rpi_axi_pmu_read().
[ ... ]
> +static void rpi_axi_pmu_del(struct perf_event *event, int flags)
> +{
> + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> + enum monitor mon = config_to_monitor(event->attr.config);
> + unsigned long spinflags;
> + int idx = event->hw.idx;
> +
> + if (idx < 0)
> + return;
> +
> + rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
> +
> + spin_lock_irqsave(&pmu->lock, spinflags);
[ ... ]
> + event->hw.idx = -1;
> + pmu->active_events--;
> + if (mon == MON__VPU)
> + pmu->active_vpu_events--;
> +
> + if (pmu->active_events == 0) {
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> + hrtimer_cancel(&pmu->hrtimer);
> + spin_lock_irqsave(&pmu->lock, spinflags);
[Severity: High]
Can this sequence cause a deadlock?
The rpi_axi_pmu_del() function can be invoked in hardirq context
via an SMP cross-CPU IPI. Since the hrtimer is configured with
HRTIMER_MODE_REL_SOFT, its callback runs in softirq context on the
same CPU. If the IPI interrupts the executing softirq timer handler,
synchronously calling hrtimer_cancel() will spin indefinitely
waiting for the preempted callback to finish.
> + if (pmu->active_events > 0)
> + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
> + HRTIMER_MODE_REL_SOFT);
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> + return;
> + }
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> +}
[ ... ]
> +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct device_node *fw_node;
> + int ret;
> +
> + spin_lock_init(&pmu->lock);
> + mutex_init(&pmu->vpu_mutex);
> +
> + pmu->chip = CHIP_BCM2835;
> +
> + pmu->pmu = (struct pmu) {
[ ... ]
> + };
> + pmu->pdev = pdev;
> + pmu->cpu = raw_smp_processor_id();
> + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
> + HRTIMER_MODE_REL_SOFT);
> + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
[ ... ]
> + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
> + if (ret) {
> + dev_err(dev, "PMU register failed %d\n", ret);
> + goto err_teardown;
> + }
> +
> + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
[Severity: Medium]
Is there a race condition here with CPU hotplug?
The CPU is assigned using raw_smp_processor_id() in a preemptible context
before cpuhp_state_add_instance() is called. If this CPU is taken offline
before the hotplug instance is registered, the subsystem misses the
transition. The teardown callback won't run, leaving the PMU permanently
bound to an offline CPU.
> + if (ret) {
> + dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
> + goto err_pmu_unregister;
> + }
> + return 0;
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812002728.2283079-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 0:40 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 0:40 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Implicit and fragile pointer cast from `struct pmu *` to `struct rpi_axi_pmu *` instead of using `container_of`.
--
--- Patch [2]: [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support ---
commit 313acedcd7c09e03b7e84ac26f169991353b78d9
Author: Ian Rogers <irogers@google.com>
Subject: perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
This commit expands the Raspberry Pi AXI PMU driver to support the Broadcom
BCM2712 SoC used in the Raspberry Pi 5. It adds new event definitions,
bus enumerations, and filter aliases specifically for the new architecture.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> index 1923dc34cb3e9..478d58c7e4f41 100644
> --- a/drivers/perf/rpi_axi_pmu.c
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> @@ -867,12 +1043,106 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
[ ... ]
> +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
> + struct attribute *attr, int unused)
> +{
> + struct device *dev = kobj_to_dev(kobj);
> + struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
[Severity: Low]
Since the perf core sets the device's driver data to a struct pmu pointer via
pmu_dev_alloc(), doesn't dev_get_drvdata(dev) return a struct pmu * here?
While this implicit cast works right now because the pmu member is located at
offset 0 in struct rpi_axi_pmu, would it be safer to use the existing
pmu_to_rpi_axi_pmu() helper to avoid potential type confusion and memory
corruption if the struct layout ever changes in the future?
> + struct perf_pmu_events_attr *pmu_attr;
> +
> + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
> +
> + if (pmu->chip == CHIP_BCM2712) {
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812002728.2283079-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 5:24 ` Ian Rogers
2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (3 more replies)
2 siblings, 4 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 5:24 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context,
ensuring spinlock acquisitions are 100% PREEMPT_RT safe.
- Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while
active perf events exist.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Changes in v4
-------------
- Global Monitor Reset Protection: In rpi_axi_pmu_enable_bus_watcher(), only issue
global monitor resets (GEN_CTL_RESET_BIT) when num_monitored == 1. For subsequent
watchers, reset only the individual bus watcher, keeping running watchers intact.
- IPC Failure Handling: In rpi_axi_pmu_read_counter(), return 0 on rpi_firmware_property()
failure instead of uninitialized -1, preventing 0xFFFFFFFF phantom counter spikes.
- Lockdep & MMIO Safety: In rpi_axi_pmu_vpu_work_handler(), only unlock spinlock during
counter reads if use_mailbox_interface is true, keeping MMIO reads lock-protected.
- Hardirq IPI Deadlock Fix: In rpi_axi_pmu_del(), use hrtimer_try_to_cancel() when in_hardirq()
to avoid infinite spinning if an SMP cross-CPU IPI interrupts a running softirq timer callback.
- Timer Handler Early Return: In rpi_axi_pmu_timer_handler(), return HRTIMER_NORESTART early
if active_events == 0.
- CPU Hotplug Preemption Race Fix: Initialize pmu->cpu = cpumask_first(cpu_online_mask) in
rpi_axi_pmu__init() to guarantee assignment to a valid online CPU core.
- Type-Safe Driver Data Extraction: In rpi_axi_pmu_events_is_visible() and cpumask_show(),
retrieve struct pmu * from dev_get_drvdata(dev) and convert via pmu_to_rpi_axi_pmu().
- Backward Compatibility: Added LINUX_VERSION_CODE check for hrtimer_setup fallback wrapper,
allowing out-of-tree builds to compile cleanly on Linux 6.12 LTS kernels.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2049 ++++++++++++++++++++++++++++++++++++
3 files changed, 2060 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-12 5:24 ` Ian Rogers
2026-08-12 5:36 ` sashiko-bot
2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
` (2 subsequent siblings)
3 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 5:24 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Add an uncore performance monitoring unit (PMU) driver for Broadcom
BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware mailbox
monitors found on Raspberry Pi 1 through 4.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1773 ++++++++++++++++++++++++++++++++++++
3 files changed, 1784 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..97c8bf34f463 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..827656b14d85
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1773 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (pmu->monitor[mon].hw_events.num_monitored == 1) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ return 0;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE) {
+ rpi_axi_pmu_read(event);
+ event->hw.state |= PERF_HES_UPTODATE;
+ }
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ if (mon == MON__SYSTEM)
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0)
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ if (in_hardirq())
+ hrtimer_try_to_cancel(&pmu->hrtimer);
+ else
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = CHIP_BCM2835;
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_pmu_unregister;
+ }
+ return 0;
+
+err_pmu_unregister:
+ perf_pmu_unregister(&pmu->pmu);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ perf_pmu_unregister(&pmu->pmu);
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 5:24 ` Ian Rogers
2026-08-12 5:34 ` sashiko-bot
2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
3 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 5:24 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
Assisted-by: Antigravity:gemini-3.6-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 2 +-
drivers/perf/rpi_axi_pmu.c | 296 +++++++++++++++++++++++++++++++++++--
2 files changed, 287 insertions(+), 11 deletions(-)
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 97c8bf34f463..84ffa65c7b79 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU
select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
help
Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
- SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
If compiled as a module, it will be called rpi_axi_pmu.
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index 827656b14d85..02ab73be70b2 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,12 +6,13 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
* The Broadcom AXI performance hardware provides up to two independent monitors:
* 1. System Monitor (MON__SYSTEM = 0):
- * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
* Directly memory-mapped via ARM physical IO memory space (MMIO).
* Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
*
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,44 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
* @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
@@ -258,6 +300,94 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
#define GEN_CTL_ENABLE_BIT BIT(0)
@@ -487,16 +617,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
-
return counter < CNT__MAX;
}
@@ -723,6 +861,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -879,12 +1055,107 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ strstr(attr->name, "jpeg_") ||
+ strstr(attr->name, "peripheral_") ||
+ strstr(attr->name, "cpu_uc_") ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_"))
+ return 0;
+ }
+
+ return attr->mode;
+}
+
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -897,9 +1168,10 @@ static const struct attribute_group rpi_axi_pmu_events_group = {
*/
static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
{
- struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
- return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
}
static DEVICE_ATTR_RO(cpumask);
@@ -1575,7 +1847,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
spin_lock_init(&pmu->lock);
mutex_init(&pmu->vpu_mutex);
- pmu->chip = CHIP_BCM2835;
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
pmu->pmu = (struct pmu) {
.module = THIS_MODULE,
@@ -1718,6 +1990,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 5:34 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 5:34 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [Medium] The BCM2712 visibility filter incorrectly hides the JPEG hardware acceleration bus PMU events.
- [Medium] Missing and mismatched PMU event aliases for BCM2712 PERIPHERAL, CPU_UC, and CPU_L2 buses.
Pre-existing issues:
- [High] PMU is exposed to userspace before driver initialization completes, allowing a Use-After-Free if a subsequent step fails.
--
--- Patch [2]: [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support ---
commit ac1314c26eb484b245288019eb6f6293df532017
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> index 827656b14d85d..02ab73be70b25 100644
> --- a/drivers/perf/rpi_axi_pmu.c
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> @@ -723,6 +861,44 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
[ ... ]
> +/*
> + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
> + * (bus=14, BCM2712_SB__CPU_L2)
> + */
> +/* BCM2712 Master ID Filtered Events */
> +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=5,counter=4,filter=10");
> +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=5,counter=2,filter=10");
[Severity: Medium]
Does this mapping correctly reflect the intended hardware bus?
The comment states that the Cortex-A76 Cores L2 cache events correspond to
bus 14 (BCM2712_SB__CPU_L2), but the event strings specify bus=5
(BCM2712_SB__SYSTEM_L2). Will this yield incorrect measurements for the CPU
L2 cache when queried via sysfs?
[ ... ]
> @@ -879,12 +1055,107 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
[ ... ]
> +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
> + struct attribute *attr, int unused)
> +{
[ ... ]
> + if (rpi_pmu->chip == CHIP_BCM2712) {
> + /*
> + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
> + * and legacy RPi 1-4 System Monitor aliases
> + */
> + if (strstr(pmu_attr->event_str, "monitor=1") ||
[ ... ]
> + strstr(attr->name, "v3d_") ||
> + strstr(attr->name, "jpeg_") ||
[Severity: Medium]
Should the jpeg_ aliases be hidden on BCM2712?
The BCM2712 hardware definitions (BCM2712_SB__JPEG = 2) indicate that the
JPEG hardware acceleration bus is supported, but this logic explicitly
hides it from userspace on this platform. Does this incorrectly deny access
to valid hardware events?
> + strstr(attr->name, "peripheral_") ||
> + strstr(attr->name, "cpu_uc_") ||
> + strstr(attr->name, "cpu_l2_") ||
[Severity: Medium]
Is it intentional to hide these legacy aliases without providing replacements
for BCM2712?
The aliases for peripheral_, cpu_uc_, and cpu_l2_ are hidden here, which
limits functionality on the new platform since equivalents weren't added in
the newly defined BCM2712 expanded event aliases.
[ ... ]
> @@ -1575,7 +1847,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
> spin_lock_init(&pmu->lock);
> mutex_init(&pmu->vpu_mutex);
>
> - pmu->chip = CHIP_BCM2835;
> + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
>
> pmu->pmu = (struct pmu) {
> .module = THIS_MODULE,
[Severity: High]
This is a pre-existing issue, but does this function expose the PMU to
userspace before driver initialization is complete?
Looking at rpi_axi_pmu__init(), the PMU is registered before ensuring all
initializations succeed:
drivers/perf/rpi_axi_pmu.c:rpi_axi_pmu__init() {
...
ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
if (ret) {
...
}
ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
if (ret) {
dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
goto err_pmu_unregister;
}
...
}
If cpuhp_state_add_instance() fails, the error path calls
perf_pmu_unregister(). However, if userspace opens the event during this
window, it will retain a reference to the struct pmu memory. That memory
gets freed by the devres framework when the probe fails, potentially causing
a use-after-free when userspace interacts with the open event.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812052417.2361040-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 5:36 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 5:36 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
- [High] Transient IPC mailbox errors corrupt PMU counter values with massive positive spikes.
- [High] Module unload triggers a Use-After-Free in CPU hotplug callback.
- [High] Global hardware monitor is never enabled for batched or grouped event additions.
- [High] Concurrent timer manipulation triggers WARN_ON_ONCE and timer failure.
- [Medium] Lockless modification of event->hw.state overwrites PERF_HES_STOPPED flag.
- [Medium] VPU hardware monitor events fail to collect data for up to 2 seconds due to missing explicit scheduling.
- [Medium] Unconditional lock drop in worker thread causes lockdep splat and unlocked MMIO access on fallback paths.
--
--- Patch [1]: [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit 8696502c11139b0621905345c84a98e3653345a3
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
> +{
> + perf_pmu_unregister(&pmu->pmu);
> + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
[Severity: High]
The perf_pmu_unregister() call frees the PMU structures, but
cpuhp_state_remove_instance() will subsequently invoke
rpi_axi_pmu_offline_cpu() on active CPUs.
In rpi_axi_pmu_offline_cpu(), we see:
perf_pmu_migrate_context(&pmu->pmu, cpu, target);
Could this cause a use-after-free by dereferencing pmu->pmu after it has
been unregistered? Should the cpuhp instance be removed before unregistering
the PMU?
[ ... ]
> +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
> + int idx, int bus, int filter)
> +{
> + int bus_control;
> +
> + if (pmu->monitor[mon].hw_events.enabled[idx])
> + return;
> +
> + bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
> + if (filter) {
> + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
> + bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
> + }
> + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
> + set_bus_watcher_control(pmu, mon, idx, bus_control);
> +
> + if (pmu->monitor[mon].hw_events.num_monitored == 1) {
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
> + }
> +}
[Severity: High]
If grouped events are added, rpi_axi_pmu_add() will increment
num_monitored multiple times before rpi_axi_pmu_start() is called for
the group.
When start() invokes this function, num_monitored could already be greater
than 1. Does this mean the global monitor control will never be enabled for
grouped event additions, leaving the hardware disabled?
[ ... ]
> +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
> + enum counter counter)
> +{
[ ... ]
> + err = rpi_firmware_property(pmu->firmware,
> + RPI_FIRMWARE_GET_PERIPH_REG,
> + tmp, sizeof(tmp));
> +
> + if (err < 0 || tmp[1] != 1) {
> + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
> + return 0;
> + }
> + ret = tmp[2] & 0x7FFFFFFF;
[Severity: High]
If the mailbox communication fails, this function returns 0. Over in
rpi_axi_pmu_vpu_work_handler(), this 0 is blindly used to calculate the new
count:
new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
...
prev_count = local64_read(&event->hw.prev_count);
delta = (new_count - prev_count) & 0x7FFFFFFF;
local64_add(delta, &event->count);
Can this lead to a massive positive spike in the counter value when
transient IPC errors occur, since 0 minus prev_count evaluates to a huge
positive delta?
[ ... ]
> +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
> +{
[ ... ]
> + /* If VPU bus watcher is not enabled on hardware, enable it in process context */
> + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
> + int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
> + int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
> +
> + spin_unlock_irq(&pmu->lock);
> + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
> + spin_lock_irq(&pmu->lock);
> + }
[Severity: Medium]
This drops the spinlock unconditionally. However, if the VPU monitor is
falling back to direct MMIO where use_mailbox_interface is false,
rpi_axi_pmu_enable_bus_watcher() calls set_bus_watcher_control() which
asserts that pmu->lock is held.
Does this lock drop violate the lockdep assertion and risk data races on
non-firmware platforms?
[ ... ]
> +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
> +{
[ ... ]
> + if (pmu->active_vpu_events > 0)
> + schedule_work(&pmu->vpu_work);
> + spin_unlock_irqrestore(&pmu->lock, flags);
> +
> + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
> + return HRTIMER_RESTART;
> +}
[Severity: High]
The spinlock is dropped before calling hrtimer_forward_now(). If a
concurrent delete and add operation occurs on another CPU during this
window, could hrtimer_start() be called while the timer is still processing?
If hrtimer_start() enqueues the timer concurrently, won't
hrtimer_forward_now() run on an already queued timer and trigger a warning
in the hrtimer core?
[ ... ]
> +static void rpi_axi_pmu_start(struct perf_event *event, int flags)
> +{
> + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> + enum monitor mon = config_to_monitor(event->attr.config);
> + enum counter counter = config_to_counter(event->attr.config);
> + unsigned long spinflags;
> +
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + event->hw.state = 0;
> + pmu->monitor[mon].hw_events.num_started++;
> + if (mon == MON__SYSTEM) {
> + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
[Severity: Medium]
When mon is MON__VPU, the hardware enablement is delegated entirely to
pmu->vpu_work. However, schedule_work() is not explicitly called here.
Does this mean VPU monitoring relies entirely on the periodic 2-second
hrtimer to start collecting data?
If so, won't short-lived perf workloads report 0 counts for VPU events
because the hardware takes up to 2 seconds to be enabled?
[ ... ]
> +static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
> +{
[ ... ]
> + if (event->hw.state & PERF_HES_STOPPED)
> + return;
> +
> + if (flags & PERF_EF_UPDATE) {
> + rpi_axi_pmu_read(event);
> + event->hw.state |= PERF_HES_UPTODATE;
> + }
> +
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + event->hw.state |= PERF_HES_STOPPED;
[Severity: Medium]
rpi_axi_pmu_stop() modifies event->hw.state to add PERF_HES_UPTODATE
without holding the spinlock.
Meanwhile, rpi_axi_pmu_vpu_work_handler() performs read-modify-write
operations on event->hw.state under the lock, such as:
event->hw.state |= PERF_HES_UPTODATE;
If stop() sets PERF_HES_STOPPED under the lock shortly after the worker
reads the old state, will the worker thread overwrite the state and
accidentally clear the PERF_HES_STOPPED flag when it writes back?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812052417.2361040-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 8:27 ` Will Deacon
2026-08-12 13:38 ` Ian Rogers
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
3 siblings, 1 reply; 59+ messages in thread
From: Will Deacon @ 2026-08-12 8:27 UTC (permalink / raw)
To: Ian Rogers
Cc: linux-perf-users, linux-rpi-kernel, linux-arm-kernel,
linux-kernel, mark.rutland, u.kleine-koenig
On Tue, Aug 11, 2026 at 10:24:15PM -0700, Ian Rogers wrote:
> This patch series adds an uncore Performance Monitoring Unit (PMU) driver
> for Broadcom AXI system and VideoCore VPU performance monitors found on
> Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Why are you sending four versions of this patch series, at -rc7, each in
reply to the previous one? Nobody is going to review that.
Will
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon
@ 2026-08-12 13:38 ` Ian Rogers
2026-08-12 16:26 ` Uwe Kleine-König
0 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 13:38 UTC (permalink / raw)
To: Will Deacon
Cc: linux-perf-users, linux-rpi-kernel, linux-arm-kernel,
linux-kernel, mark.rutland, u.kleine-koenig
On Wed, Aug 12, 2026 at 1:27 AM Will Deacon <will@kernel.org> wrote:
>
> On Tue, Aug 11, 2026 at 10:24:15PM -0700, Ian Rogers wrote:
> > This patch series adds an uncore Performance Monitoring Unit (PMU) driver
> > for Broadcom AXI system and VideoCore VPU performance monitors found on
> > Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
>
> Why are you sending four versions of this patch series, at -rc7, each in
> reply to the previous one? Nobody is going to review that.
See the cover letter for changes. They address the sashiko reviews,
you may not see these reviews as they are only sent to me and
linux-perf-users, which is somewhat customary in the sashiko setup.
I don't see any relevance in rc7, I'm just mailing out a new driver
now that I have the opportunity to look at it. Whether and when it
gets pulled upstream is up to a maintainer. Another useful series to
pull would be (imo):
https://lore.kernel.org/linux-perf-users/20260101-apple-cpmu-v10-0-48812c529ffc@gmail.com/
The changes are sent in reply to each other because the entire series
is visible in one mailbox (b4 mbox), which I find easier to work with
and provides me better context. Others use semcode to do something
similar, but I've not explored that.
Let me know if I should drop you from the cc, I placed you there as
Mark and you were the only suggested reviewers from
scripts/get_maintainer.pl but I don't want to spam uninterested
people.
Thanks,
Ian
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 13:38 ` Ian Rogers
@ 2026-08-12 16:26 ` Uwe Kleine-König
2026-08-12 21:37 ` Ian Rogers
0 siblings, 1 reply; 59+ messages in thread
From: Uwe Kleine-König @ 2026-08-12 16:26 UTC (permalink / raw)
To: Ian Rogers
Cc: Will Deacon, linux-perf-users, linux-rpi-kernel, linux-arm-kernel,
linux-kernel, mark.rutland
[-- Attachment #1: Type: text/plain, Size: 595 bytes --]
Hello Ian,
On Wed, Aug 12, 2026 at 06:38:24AM -0700, Ian Rogers wrote:
> The changes are sent in reply to each other because the entire series
> is visible in one mailbox (b4 mbox), which I find easier to work with
> and provides me better context. Others use semcode to do something
> similar, but I've not explored that.
The usual recommendation is to start a new thread for a new revision.
Depending on the mail reader (configuration) new mails to an old thread
might or might not appear at the end of people's mailbox where new mail
usually lands and so might be missed.
Best regards
Uwe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 16:26 ` Uwe Kleine-König
@ 2026-08-12 21:37 ` Ian Rogers
2026-08-13 6:59 ` Uwe Kleine-König
0 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 21:37 UTC (permalink / raw)
To: Uwe Kleine-König
Cc: Will Deacon, linux-perf-users, linux-rpi-kernel, linux-arm-kernel,
linux-kernel, mark.rutland
On Wed, Aug 12, 2026 at 9:26 AM Uwe Kleine-König
<u.kleine-koenig@baylibre.com> wrote:
>
> Hello Ian,
>
> On Wed, Aug 12, 2026 at 06:38:24AM -0700, Ian Rogers wrote:
> > The changes are sent in reply to each other because the entire series
> > is visible in one mailbox (b4 mbox), which I find easier to work with
> > and provides me better context. Others use semcode to do something
> > similar, but I've not explored that.
>
> The usual recommendation is to start a new thread for a new revision.
> Depending on the mail reader (configuration) new mails to an old thread
> might or might not appear at the end of people's mailbox where new mail
> usually lands and so might be missed.
Yep, I agree. People who configure their email client so that a thread
doesn't move to the top of their inbox upon receiving a reply are
prone to missing subsequent replies in that thread. I suggest that
people in that position change their email configuration, or use
lore.kernel.org, which doesn't suffer from this bad configuration, for
example:
https://lore.kernel.org/linux-perf-users/20260811083828.2057695-1-irogers@google.com/
Thanks,
Ian
> Best regards
> Uwe
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v5 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
` (2 preceding siblings ...)
2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon
@ 2026-08-12 23:01 ` Ian Rogers
2026-08-12 23:01 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (4 more replies)
3 siblings, 5 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:01 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context,
ensuring spinlock acquisitions are 100% PREEMPT_RT safe.
- Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while
active perf events exist.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Changes in v5
-------------
- Teardown UAF/Orphaning Fix: Reversed module exit logic, calling cpuhp_state_remove_instance()
*before* perf_pmu_unregister() to prevent accessing an unregistered PMU in the offline
CPU handler during device removal, and to prevent event orphaning races during hotplug.
- VPU Hardware Watcher Leak Fix: In rpi_axi_pmu_del(), hardware disablement operations for VPU
bus watchers are now properly deferred to the vpu_work workqueue to avoid silently leaking
running hardware counters when the VPU monitor is no longer in use.
- Grouped Event Start Bug Fix: Deprecated reliance on num_monitored == 1 to initialize
the global bus watcher. Instead, use a dedicated monitor_running flag correctly to
start the global monitor when the first bus watcher gets enabled, mitigating bugs when
adding grouped PMU events.
- IPC Phantom Spikes Zero Fallback Fix: Modified rpi_axi_pmu_read_counter() to return U32_MAX
on IPC or MMIO hardware read failures, gracefully ignoring the delta calculation returning.
This fixes unsigned 31-bit huge spikes where delta (0 - old_count) & 0x7FFFFFFF occurred.
- Code Comments: Added doc block comments inside rpi_axi_pmu_read(), rpi_axi_pmu_vpu_work_handler(),
and everywhere U32_MAX returns occur to document the explicit justification for why U32_MAX indicates
failure (valid counters are 31-bit) and why the driver drops the sample to avoid artificial spikes.
- VPU Data Race Lockdep Fix: Rewrote locking hierarchy inside vpu_work_handler(). PMU spinlock
is now correctly held across rpi_axi_pmu_enable_bus_watcher() unconditionally since the function
touches active monitor registers directly via MMIO, dropping it purely for mailbox IPC reads.
- Hrtimer Add/Del Forward Concurrency Warning Fix: rpi_axi_pmu_timer_handler() now executes
hrtimer_forward_now() *before* releasing the PMU spinlock, fully mitigating timer enqueue races
resulting from a simultaneous parallel pmu->add()/pmu->del().
- VPU Start Asynchronous Latency: Added explicit schedule_work(&pmu->vpu_work) call inside
rpi_axi_pmu_start() when starting VPU-tracked events so they begin immediately instead of
waiting for a 2-second timer interrupt period for short workloads.
- Event Stop Read-Modify-Write Spinlock Protection: Moved event->hw.state modifications inside
rpi_axi_pmu_stop() under the local CPU PMU irqsave spinlock to safeguard atomic bitwise flags
against the timer handler or workqueue updates.
- BCM2712 Pre-probe Initializer UAF race: Shifted perf_pmu_register() to the very end of
rpi_axi_pmu__init() to ensure all hardware structures and CPU HP bindings securely exist prior to
exposing the PMU to userspace tools preventing devres unregistration memory faults.
- BCM2712 ARM CPU L2 Event Correction: Corrected the bus index for the Cortex-A76 Cores PMU event
aliases to bus=14 (BCM2712_SB__CPU_L2) to match architectural routing specifications on the RPi 5.
- BCM2712 JPEG Bus Monitor Visibility: Stopped improperly hiding the legacy JPEG bus aliases in the
is_visible() callback on BCM2712 chips.
- BCM2712 Custom Event Replacements: Created specific BCM2712 equivalents for peripheral_rtrans/wtrans
and cpu_uc_rtrans/wtrans since raw bus mappings shifted structurally on VideoCore VII.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2078 ++++++++++++++++++++++++++++++++++++
3 files changed, 2089 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
@ 2026-08-12 23:01 ` Ian Rogers
2026-08-12 23:15 ` sashiko-bot
2026-08-12 23:01 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
` (3 subsequent siblings)
4 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:01 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
TAG=agy
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1793 ++++++++++++++++++++++++++++++++++++
3 files changed, 1804 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..97c8bf34f463 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..e5cc86fc78b2
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1793 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ if (mon == MON__SYSTEM)
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ if (in_hardirq())
+ hrtimer_try_to_cancel(&pmu->hrtimer);
+ else
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = CHIP_BCM2835;
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp_remove;
+ }
+
+ return 0;
+
+err_cpuhp_remove:
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ perf_pmu_unregister(&pmu->pmu);
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
2026-08-12 23:01 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 23:01 ` Ian Rogers
2026-08-12 23:18 ` sashiko-bot
2026-08-12 23:07 ` [PATCH v5 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
` (2 subsequent siblings)
4 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:01 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
TAG=agy
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 2 +-
drivers/perf/rpi_axi_pmu.c | 365 +++++++++++++++++++++++++++++++++++--
2 files changed, 349 insertions(+), 18 deletions(-)
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 97c8bf34f463..84ffa65c7b79 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU
select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
help
Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
- SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
If compiled as a module, it will be called rpi_axi_pmu.
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index e5cc86fc78b2..a24b6587f871 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,12 +6,13 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
* The Broadcom AXI performance hardware provides up to two independent monitors:
* 1. System Monitor (MON__SYSTEM = 0):
- * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
* Directly memory-mapped via ARM physical IO memory space (MMIO).
* Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
*
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,44 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
* @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
@@ -258,6 +300,94 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
#define GEN_CTL_ENABLE_BIT BIT(0)
@@ -367,6 +497,7 @@ struct rpi_axi_hw_events {
int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
int num_monitored;
int num_started;
+bool monitor_running;
bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
};
@@ -488,16 +619,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
-
return counter < CNT__MAX;
}
@@ -724,6 +863,49 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans, "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans, "monitor=0,bus=13,counter=2");
+
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -880,12 +1062,111 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ strstr(attr->name, "peripheral_") ||
+ strstr(attr->name, "cpu_uc_") ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_"))
+ return 0;
+ }
+
+ return attr->mode;
+}
+
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -898,9 +1179,10 @@ static const struct attribute_group rpi_axi_pmu_events_group = {
*/
static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
{
- struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
- return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
}
static DEVICE_ATTR_RO(cpumask);
@@ -1177,6 +1459,11 @@ static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, i
if (err < 0 || tmp[1] != 1) {
dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
return U32_MAX;
}
ret = tmp[2] & 0x7FFFFFFF;
@@ -1222,6 +1509,10 @@ static void rpi_axi_pmu_read(struct perf_event *event)
prev_count = local64_read(&event->hw.prev_count);
new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
spin_unlock_irqrestore(&pmu->lock, flags);
return;
}
@@ -1287,6 +1578,10 @@ static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
}
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
if (new_count == U32_MAX)
continue;
@@ -1311,6 +1606,31 @@ static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
}
}
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.refcount[idx] == 0 &&
+ pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
spin_unlock_irq(&pmu->lock);
mutex_unlock(&pmu->vpu_mutex);
}
@@ -1349,6 +1669,10 @@ static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
prev_count = local64_read(&event->hw.prev_count);
new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
if (new_count != U32_MAX) {
local64_set(&event->hw.prev_count, new_count);
delta = (new_count - prev_count) & 0x7FFFFFFF;
@@ -1507,15 +1831,17 @@ static void rpi_axi_pmu_del(struct perf_event *event, int flags)
if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
pmu->monitor[mon].hw_events.refcount[idx]--;
if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
- if (mon == MON__SYSTEM)
- rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
- pmu->monitor[mon].hw_events.enabled[idx] = false;
pmu->monitor[mon].hw_events.num_monitored--;
- if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
- set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
- pmu->monitor[mon].hw_events.monitor_running = false;
+ if (mon == MON__SYSTEM) {
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
}
}
}
@@ -1594,7 +1920,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
spin_lock_init(&pmu->lock);
mutex_init(&pmu->vpu_mutex);
- pmu->chip = CHIP_BCM2835;
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
pmu->pmu = (struct pmu) {
.module = THIS_MODULE,
@@ -1688,8 +2014,9 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
*/
static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
{
- perf_pmu_unregister(&pmu->pmu);
cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+
+ perf_pmu_unregister(&pmu->pmu);
hrtimer_cancel(&pmu->hrtimer);
cancel_work_sync(&pmu->vpu_work);
if (pmu->firmware)
@@ -1738,6 +2065,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v5 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
2026-08-12 23:01 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 23:01 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 23:07 ` Ian Rogers
2026-08-12 23:07 ` Ian Rogers
2026-08-13 4:35 ` [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
4 siblings, 0 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:07 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
On Wed, Aug 12, 2026 at 4:01 PM Ian Rogers <irogers@google.com> wrote:
>
> This patch series adds an uncore Performance Monitoring Unit (PMU) driver
> for Broadcom AXI system and VideoCore VPU performance monitors found on
> Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
>
> Motivation & Background
> ------------------------
> Currently, Linux lacks a standard perf-API compatible driver for the
> Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
> out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
> kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
> `perf list`, etc.).
>
> This driver implements standard `struct pmu` hardware uncore callbacks
> under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
> scaling (`Bytes`), and bus filtering directly to user space.
>
> Key Architectural Improvements & Features
> ------------------------------------------
> 1. Standard Linux Perf Integration:
> - Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
> - Supports event sampling and hardware counter accumulation (`local64_add`),
> automatically managing 31-bit hardware counter wraparound across high-bandwidth
> interconnect transfers.
>
> 2. CPU Hotplug Support (`cpuhp`):
> - Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
> - Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
> CPU core when a designated CPU goes offline, avoiding stale uncore state.
>
> 3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
> - System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
> directly mapped over ARM physical memory space (`MON__SYSTEM`).
> - VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
> VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
> workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
> sleeps or blocking in timer/interrupt context.
>
> 4. PREEMPT_RT & Safety Hardening:
> - Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context,
> ensuring spinlock acquisitions are 100% PREEMPT_RT safe.
> - Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while
> active perf events exist.
>
> 5. SoC Generation Support:
> - Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
> - Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
> Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
> interconnect monitoring.
>
> Hardware Validation
> -------------------
> The driver has been validated on real hardware across multiple SoC generations:
> - Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
> firmware mailbox IPC performance counters.
> - Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
> refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
> Southbridge transfers.
>
> Changes in v5
> -------------
> - Teardown UAF/Orphaning Fix: Reversed module exit logic, calling cpuhp_state_remove_instance()
> *before* perf_pmu_unregister() to prevent accessing an unregistered PMU in the offline
> CPU handler during device removal, and to prevent event orphaning races during hotplug.
> - VPU Hardware Watcher Leak Fix: In rpi_axi_pmu_del(), hardware disablement operations for VPU
> bus watchers are now properly deferred to the vpu_work workqueue to avoid silently leaking
> running hardware counters when the VPU monitor is no longer in use.
> - Grouped Event Start Bug Fix: Deprecated reliance on num_monitored == 1 to initialize
> the global bus watcher. Instead, use a dedicated monitor_running flag correctly to
> start the global monitor when the first bus watcher gets enabled, mitigating bugs when
> adding grouped PMU events.
> - IPC Phantom Spikes Zero Fallback Fix: Modified rpi_axi_pmu_read_counter() to return U32_MAX
> on IPC or MMIO hardware read failures, gracefully ignoring the delta calculation returning.
> This fixes unsigned 31-bit huge spikes where delta (0 - old_count) & 0x7FFFFFFF occurred.
> - Code Comments: Added doc block comments inside rpi_axi_pmu_read(), rpi_axi_pmu_vpu_work_handler(),
> and everywhere U32_MAX returns occur to document the explicit justification for why U32_MAX indicates
> failure (valid counters are 31-bit) and why the driver drops the sample to avoid artificial spikes.
> - VPU Data Race Lockdep Fix: Rewrote locking hierarchy inside vpu_work_handler(). PMU spinlock
> is now correctly held across rpi_axi_pmu_enable_bus_watcher() unconditionally since the function
> touches active monitor registers directly via MMIO, dropping it purely for mailbox IPC reads.
> - Hrtimer Add/Del Forward Concurrency Warning Fix: rpi_axi_pmu_timer_handler() now executes
> hrtimer_forward_now() *before* releasing the PMU spinlock, fully mitigating timer enqueue races
> resulting from a simultaneous parallel pmu->add()/pmu->del().
> - VPU Start Asynchronous Latency: Added explicit schedule_work(&pmu->vpu_work) call inside
> rpi_axi_pmu_start() when starting VPU-tracked events so they begin immediately instead of
> waiting for a 2-second timer interrupt period for short workloads.
> - Event Stop Read-Modify-Write Spinlock Protection: Moved event->hw.state modifications inside
> rpi_axi_pmu_stop() under the local CPU PMU irqsave spinlock to safeguard atomic bitwise flags
> against the timer handler or workqueue updates.
> - BCM2712 Pre-probe Initializer UAF race: Shifted perf_pmu_register() to the very end of
> rpi_axi_pmu__init() to ensure all hardware structures and CPU HP bindings securely exist prior to
> exposing the PMU to userspace tools preventing devres unregistration memory faults.
> - BCM2712 ARM CPU L2 Event Correction: Corrected the bus index for the Cortex-A76 Cores PMU event
> aliases to bus=14 (BCM2712_SB__CPU_L2) to match architectural routing specifications on the RPi 5.
> - BCM2712 JPEG Bus Monitor Visibility: Stopped improperly hiding the legacy JPEG bus aliases in the
> is_visible() callback on BCM2712 chips.
> - BCM2712 Custom Event Replacements: Created specific BCM2712 equivalents for peripheral_rtrans/wtrans
> and cpu_uc_rtrans/wtrans since raw bus mappings shifted structurally on VideoCore VII.
Sorry, inadvertently added TAG=agy. Will resend with this removed.
Thanks,
Ian
> Ian Rogers (2):
> perf: Add Raspberry Pi BCM2835 AXI PMU driver
> perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
>
> drivers/perf/Kconfig | 10 +
> drivers/perf/Makefile | 1 +
> drivers/perf/rpi_axi_pmu.c | 2078 ++++++++++++++++++++++++++++++++++++
> 3 files changed, 2089 insertions(+)
> create mode 100644 drivers/perf/rpi_axi_pmu.c
>
> --
> 2.55.0.679.g6767b8d81c-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v5 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
` (2 preceding siblings ...)
2026-08-12 23:07 ` [PATCH v5 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-12 23:07 ` Ian Rogers
2026-08-12 23:07 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 23:08 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 4:35 ` [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
4 siblings, 2 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:07 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context,
ensuring spinlock acquisitions are 100% PREEMPT_RT safe.
- Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while
active perf events exist.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been validated on real hardware across multiple SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Changes in v5
-------------
- Teardown UAF/Orphaning Fix: Reversed module exit logic, calling cpuhp_state_remove_instance()
*before* perf_pmu_unregister() to prevent accessing an unregistered PMU in the offline
CPU handler during device removal, and to prevent event orphaning races during hotplug.
- VPU Hardware Watcher Leak Fix: In rpi_axi_pmu_del(), hardware disablement operations for VPU
bus watchers are now properly deferred to the vpu_work workqueue to avoid silently leaking
running hardware counters when the VPU monitor is no longer in use.
- Grouped Event Start Bug Fix: Deprecated reliance on num_monitored == 1 to initialize
the global bus watcher. Instead, use a dedicated monitor_running flag correctly to
start the global monitor when the first bus watcher gets enabled, mitigating bugs when
adding grouped PMU events.
- IPC Phantom Spikes Zero Fallback Fix: Modified rpi_axi_pmu_read_counter() to return U32_MAX
on IPC or MMIO hardware read failures, gracefully ignoring the delta calculation returning.
This fixes unsigned 31-bit huge spikes where delta (0 - old_count) & 0x7FFFFFFF occurred.
- Code Comments: Added doc block comments inside rpi_axi_pmu_read(), rpi_axi_pmu_vpu_work_handler(),
and everywhere U32_MAX returns occur to document the explicit justification for why U32_MAX indicates
failure (valid counters are 31-bit) and why the driver drops the sample to avoid artificial spikes.
- VPU Data Race Lockdep Fix: Rewrote locking hierarchy inside vpu_work_handler(). PMU spinlock
is now correctly held across rpi_axi_pmu_enable_bus_watcher() unconditionally since the function
touches active monitor registers directly via MMIO, dropping it purely for mailbox IPC reads.
- Hrtimer Add/Del Forward Concurrency Warning Fix: rpi_axi_pmu_timer_handler() now executes
hrtimer_forward_now() *before* releasing the PMU spinlock, fully mitigating timer enqueue races
resulting from a simultaneous parallel pmu->add()/pmu->del().
- VPU Start Asynchronous Latency: Added explicit schedule_work(&pmu->vpu_work) call inside
rpi_axi_pmu_start() when starting VPU-tracked events so they begin immediately instead of
waiting for a 2-second timer interrupt period for short workloads.
- Event Stop Read-Modify-Write Spinlock Protection: Moved event->hw.state modifications inside
rpi_axi_pmu_stop() under the local CPU PMU irqsave spinlock to safeguard atomic bitwise flags
against the timer handler or workqueue updates.
- BCM2712 Pre-probe Initializer UAF race: Shifted perf_pmu_register() to the very end of
rpi_axi_pmu__init() to ensure all hardware structures and CPU HP bindings securely exist prior to
exposing the PMU to userspace tools preventing devres unregistration memory faults.
- BCM2712 ARM CPU L2 Event Correction: Corrected the bus index for the Cortex-A76 Cores PMU event
aliases to bus=14 (BCM2712_SB__CPU_L2) to match architectural routing specifications on the RPi 5.
- BCM2712 JPEG Bus Monitor Visibility: Stopped improperly hiding the legacy JPEG bus aliases in the
is_visible() callback on BCM2712 chips.
- BCM2712 Custom Event Replacements: Created specific BCM2712 equivalents for peripheral_rtrans/wtrans
and cpu_uc_rtrans/wtrans since raw bus mappings shifted structurally on VideoCore VII.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2124 ++++++++++++++++++++++++++++++++++++
3 files changed, 2135 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 23:07 ` Ian Rogers
@ 2026-08-12 23:07 ` Ian Rogers
2026-08-12 23:25 ` sashiko-bot
2026-08-13 0:06 ` Florian Fainelli
2026-08-12 23:08 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
1 sibling, 2 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:07 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 1793 ++++++++++++++++++++++++++++++++++++
3 files changed, 1804 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..97c8bf34f463 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..e5cc86fc78b2
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1793 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ if (mon == MON__SYSTEM)
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.enabled[idx] = false;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ if (in_hardirq())
+ hrtimer_try_to_cancel(&pmu->hrtimer);
+ else
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = CHIP_BCM2835;
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp_remove;
+ }
+
+ return 0;
+
+err_cpuhp_remove:
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ perf_pmu_unregister(&pmu->pmu);
+ cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 23:07 ` Ian Rogers
2026-08-12 23:07 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 23:08 ` Ian Rogers
2026-08-12 23:24 ` sashiko-bot
1 sibling, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-12 23:08 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 2 +-
drivers/perf/rpi_axi_pmu.c | 365 +++++++++++++++++++++++++++++++++++--
2 files changed, 349 insertions(+), 18 deletions(-)
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 97c8bf34f463..84ffa65c7b79 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -331,7 +331,7 @@ config RASPBERRYPI_AXI_PMU
select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
help
Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
- SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711).
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
If compiled as a module, it will be called rpi_axi_pmu.
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index e5cc86fc78b2..a24b6587f871 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,12 +6,13 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
* The Broadcom AXI performance hardware provides up to two independent monitors:
* 1. System Monitor (MON__SYSTEM = 0):
- * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS).
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
* Directly memory-mapped via ARM physical IO memory space (MMIO).
* Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
*
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,44 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
* @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
@@ -258,6 +300,94 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
#define GEN_CTL_ENABLE_BIT BIT(0)
@@ -367,6 +497,7 @@ struct rpi_axi_hw_events {
int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
int num_monitored;
int num_started;
+bool monitor_running;
bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
};
@@ -488,16 +619,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
-
return counter < CNT__MAX;
}
@@ -724,6 +863,49 @@ PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtra
PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans, "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans, "monitor=0,bus=13,counter=2");
+
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -880,12 +1062,111 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ strstr(attr->name, "peripheral_") ||
+ strstr(attr->name, "cpu_uc_") ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_"))
+ return 0;
+ }
+
+ return attr->mode;
+}
+
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -898,9 +1179,10 @@ static const struct attribute_group rpi_axi_pmu_events_group = {
*/
static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
{
- struct rpi_axi_pmu *pmu = dev_get_drvdata(dev);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
- return cpumap_print_to_pagebuf(true, buf, cpumask_of(pmu->cpu));
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
}
static DEVICE_ATTR_RO(cpumask);
@@ -1177,6 +1459,11 @@ static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, i
if (err < 0 || tmp[1] != 1) {
dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
return U32_MAX;
}
ret = tmp[2] & 0x7FFFFFFF;
@@ -1222,6 +1509,10 @@ static void rpi_axi_pmu_read(struct perf_event *event)
prev_count = local64_read(&event->hw.prev_count);
new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
spin_unlock_irqrestore(&pmu->lock, flags);
return;
}
@@ -1287,6 +1578,10 @@ static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
}
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
if (new_count == U32_MAX)
continue;
@@ -1311,6 +1606,31 @@ static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
}
}
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.refcount[idx] == 0 &&
+ pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
spin_unlock_irq(&pmu->lock);
mutex_unlock(&pmu->vpu_mutex);
}
@@ -1349,6 +1669,10 @@ static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
prev_count = local64_read(&event->hw.prev_count);
new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
if (new_count != U32_MAX) {
local64_set(&event->hw.prev_count, new_count);
delta = (new_count - prev_count) & 0x7FFFFFFF;
@@ -1507,15 +1831,17 @@ static void rpi_axi_pmu_del(struct perf_event *event, int flags)
if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
pmu->monitor[mon].hw_events.refcount[idx]--;
if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
- if (mon == MON__SYSTEM)
- rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
- pmu->monitor[mon].hw_events.enabled[idx] = false;
pmu->monitor[mon].hw_events.num_monitored--;
- if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
- set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
- pmu->monitor[mon].hw_events.monitor_running = false;
+ if (mon == MON__SYSTEM) {
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
}
}
}
@@ -1594,7 +1920,7 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
spin_lock_init(&pmu->lock);
mutex_init(&pmu->vpu_mutex);
- pmu->chip = CHIP_BCM2835;
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
pmu->pmu = (struct pmu) {
.module = THIS_MODULE,
@@ -1688,8 +2014,9 @@ static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pd
*/
static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
{
- perf_pmu_unregister(&pmu->pmu);
cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+
+ perf_pmu_unregister(&pmu->pmu);
hrtimer_cancel(&pmu->hrtimer);
cancel_work_sync(&pmu->vpu_work);
if (pmu->firmware)
@@ -1738,6 +2065,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 23:01 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 23:15 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 23:15 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Use-After-Free during CPU context migration because cpuhp instance is removed after PMU unregistration.
- [Medium] Hardware VPU bus watchers are never disabled due to dead code in the deferred teardown logic.
- [Low] Compilation error due to undeclared 'monitor_running' field in 'struct rpi_axi_hw_events'.
--
--- Patch [1]: [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit 5c4597457ab7a1a4ae244f1e321c00bd28692a8a
Author: Ian Rogers <irogers@google.com>
Date: Wed Aug 12 23:01:44 2026 +0000
perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..e5cc86fc78b25
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +struct rpi_axi_hw_events {
> + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
> + int filter[NUM_BUS_WATCHERS_PER_MONITOR];
> + int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
> + int num_monitored;
> + int num_started;
> + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
> +};
[Severity: Low]
Does this code fail to compile? It looks like the monitor_running field is
accessed in rpi_axi_hw_events__init() and rpi_axi_pmu_enable_bus_watcher(),
but it is missing from the definition of struct rpi_axi_hw_events here.
> +
> +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
> +{
> + hw_events->num_monitored = 0;
> + hw_events->num_started = 0;
> + hw_events->monitor_running = false;
[ ... ]
> +static void rpi_axi_pmu_del(struct perf_event *event, int flags)
> +{
> + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> + enum monitor mon = config_to_monitor(event->attr.config);
> + unsigned long spinflags;
> + int idx = event->hw.idx;
> +
> + if (idx < 0)
> + return;
> +
> + rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
> +
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
> + if (pmu->events[i] == event) {
> + pmu->events[i] = NULL;
> + pmu->event_gen[i]++;
> + break;
> + }
> + }
> +
> + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
> + pmu->monitor[mon].hw_events.refcount[idx]--;
> + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
> + if (mon == MON__SYSTEM)
> + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
> + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
> + pmu->monitor[mon].hw_events.enabled[idx] = false;
[Severity: Medium]
Will this leave hardware VPU bus watchers running indefinitely?
When a VPU perf event is deleted, rpi_axi_pmu_del() synchronously sets
enabled[idx] = false. The background rpi_axi_pmu_vpu_work_handler() relies on
checking both refcount[idx] == 0 and enabled[idx] to perform the actual
hardware teardown over the mailbox.
Because enabled[idx] is cleared here first, that condition becomes impossible
to satisfy in the work handler. Could this lead to the VPU monitor being left
running when no events are active, wasting power and bus bandwidth?
> + pmu->monitor[mon].hw_events.num_monitored--;
> + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> + pmu->monitor[mon].hw_events.monitor_running = false;
> + }
> + }
> + }
[ ... ]
> +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
> +{
[ ... ]
> + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
> + if (ret) {
> + dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
> + goto err_teardown;
> + }
> +
> + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
> + if (ret) {
> + dev_err(dev, "PMU register failed %d\n", ret);
> + goto err_cpuhp_remove;
> + }
> +
> + return 0;
> +
> +err_cpuhp_remove:
> + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
[Severity: High]
Does this error path trigger a use-after-free or panic during CPU context
migration?
If perf_pmu_register() fails, calling cpuhp_state_remove_instance() will
trigger the rpi_axi_pmu_offline_cpu() callback for online CPUs. That
callback executes perf_pmu_migrate_context(), which will attempt to migrate
a PMU that was never successfully registered and has a NULL context pointer.
> +err_teardown:
> + hrtimer_cancel(&pmu->hrtimer);
> + cancel_work_sync(&pmu->vpu_work);
> +err_firmware_put:
> + if (pmu->firmware)
> + rpi_firmware_put(pmu->firmware);
> + return ret;
> +}
> +
> +/**
> + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
> + * @pmu: Pointer to PMU context
> + */
> +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
> +{
> + perf_pmu_unregister(&pmu->pmu);
> + cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
[Severity: High]
Can unregistering the PMU before removing the cpuhp instance cause a
use-after-free?
perf_pmu_unregister() frees the PMU's CPU context allocations. The subsequent
call to cpuhp_state_remove_instance() triggers the rpi_axi_pmu_offline_cpu()
callback for online CPUs.
This callback will then call perf_pmu_migrate_context(), which dereferences
the now-freed cpu_pmu_context, leading to a UAF. Should
cpuhp_state_remove_instance() be called before perf_pmu_unregister() to
ensure the PMU context remains valid during migration?
> + hrtimer_cancel(&pmu->hrtimer);
> + cancel_work_sync(&pmu->vpu_work);
> + if (pmu->firmware)
> + rpi_firmware_put(pmu->firmware);
> +}
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812230145.2644281-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 23:01 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 23:18 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 23:18 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Removing `enabled[idx] = false` from `rpi_axi_pmu_del` causes bus watcher misconfiguration when a watcher index is rapidly reused.
- [Medium] Overly broad substring matching inadvertently hides newly added BCM2712 events on RPi 5 while exposing them on RPi 1-4.
- [Medium] The VPU monitor hardware is not reliably disabled during module unload because `cancel_work_sync()` is used instead of `flush_work()`.
--
--- Patch [2]: [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support ---
commit 71567bad9b4b4c6115df3982a734e87c6c908033
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
This patch expands the Raspberry Pi AXI PMU driver to support Broadcom
BCM2712 system monitors. It adds new monitored buses, filter options,
and event aliases for the Raspberry Pi 5 platform.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> index e5cc86fc78b25..a24b6587f8718 100644
> --- a/drivers/perf/rpi_axi_pmu.c
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> @@ -1113,31 +1203,66 @@ static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
> struct attribute *attr, int unused)
> {
[ ... ]
> if (rpi_pmu->chip == CHIP_BCM2712) {
> /*
> * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
> * and legacy RPi 1-4 System Monitor aliases
> */
> if (strstr(pmu_attr->event_str, "monitor=1") ||
[ ... ]
> strstr(attr->name, "peripheral_") ||
> strstr(attr->name, "cpu_uc_") ||
[ ... ]
> return 0;
> } else {
> /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
> if (strstr(attr->name, "pcie_rp1") ||
[ ... ]
> return 0;
> }
>
> return attr->mode;
> }
[Severity: Medium]
Will these substring matches inadvertently hide the newly added BCM2712 events
on RPi 5?
The new BCM2712 events are named "bcm2712_peripheral_rtrans" and
"bcm2712_cpu_uc_rtrans", which will match "peripheral_" and "cpu_uc_"
and be hidden here.
Conversely, won't these same new events be mistakenly exposed on older chips
because they don't match any of the substrings in the `else` block?
[ ... ]
> @@ -1810,14 +1831,16 @@ static void rpi_axi_pmu_del(struct perf_event *event, int flags)
[ ... ]
> if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
> pmu->monitor[mon].hw_events.refcount[idx]--;
> if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
> - if (mon == MON__SYSTEM)
> - rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
> pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
> - pmu->monitor[mon].hw_events.enabled[idx] = false;
> pmu->monitor[mon].hw_events.num_monitored--;
> - if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
> - set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> - pmu->monitor[mon].hw_events.monitor_running = false;
> + if (mon == MON__SYSTEM) {
> + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> + if (pmu->monitor[mon].hw_events.num_monitored == 0) {
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> + pmu->monitor[mon].hw_events.monitor_running = false;
> + }
> + } else if (mon == MON__VPU) {
> + schedule_work(&pmu->vpu_work);
> }
> }
> }
[Severity: High]
Is it safe to remove `pmu->monitor[mon].hw_events.enabled[idx] = false` here?
If a VPU event is deleted, the watcher slot is marked as free. If a new VPU
event is added immediately before `vpu_work` runs to perform the teardown,
the new event could claim this slot.
When `vpu_work` runs, it checks `if (!enabled[idx])` to program the hardware.
Since `enabled[idx]` is still true from the deleted event, it skips
initialization, causing the hardware to monitor the old bus while the new
perf event incorrectly reports those counts.
[ ... ]
> @@ -2014,9 +2065,10 @@ static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
> {
> - perf_pmu_unregister(&pmu->pmu);
> cpuhp_state_remove_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
> +
> + perf_pmu_unregister(&pmu->pmu);
> hrtimer_cancel(&pmu->hrtimer);
> cancel_work_sync(&pmu->vpu_work);
> if (pmu->firmware)
[Severity: Medium]
Since `rpi_axi_pmu_del()` was updated to schedule `vpu_work` to disable the
VPU hardware, does this `cancel_work_sync()` prevent the hardware teardown
from completing during module unload?
`perf_pmu_unregister()` will call `rpi_axi_pmu_del()`, which schedules
the asynchronous teardown. Canceling it immediately after means the VPU AXI
bus watchers could be left running. Should this be `flush_work()` instead?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812230145.2644281-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-12 23:08 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-12 23:24 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 23:24 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] VPU bus watcher reconfiguration is skipped if a watcher slot is reallocated before asynchronous vpu_work executes, causing silent data corruption.
- [Medium] Broad substring matching in `rpi_axi_pmu_events_is_visible()` unintentionally hides newly added BCM2712-specific event aliases on the BCM2712 platform.
--
--- Patch [2]: [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support ---
commit f428ceab1c23265b486b6b953b78aae6ae47a096
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> --- a/drivers/perf/rpi_axi_pmu.c
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> @@ -880,12 +1062,111 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
[ ... ]
> +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
> + struct attribute *attr, int unused)
> +{
> + struct device *dev = kobj_to_dev(kobj);
> + struct pmu *pmu = dev_get_drvdata(dev);
> + struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
> + struct perf_pmu_events_attr *pmu_attr;
> +
> + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
> +
> + if (rpi_pmu->chip == CHIP_BCM2712) {
> + /*
> + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
> + * and legacy RPi 1-4 System Monitor aliases
> + */
> + if (strstr(pmu_attr->event_str, "monitor=1") ||
> + strstr(attr->name, "cpu0_") ||
> + strstr(attr->name, "cpu1_") ||
> + strstr(attr->name, "dma0_") ||
> + strstr(attr->name, "dma1_") ||
> + strstr(attr->name, "v3d0_") ||
> + strstr(attr->name, "v3d1_") ||
> + strstr(attr->name, "hvs_system") ||
> + strstr(attr->name, "isp_system") ||
> + strstr(attr->name, "usb_system") ||
> + strstr(attr->name, "ccp2tx_") ||
> + strstr(attr->name, "mphi_") ||
> + strstr(attr->name, "h264_") ||
> + strstr(attr->name, "v3d_") ||
> + strstr(attr->name, "peripheral_") ||
> + strstr(attr->name, "cpu_uc_") ||
[Severity: Medium]
Does this broad substring matching unintentionally hide the newly added
bcm2712_peripheral_* and bcm2712_cpu_uc_* event aliases on the BCM2712
platform?
Since the new aliases are named bcm2712_peripheral_rtrans and
bcm2712_cpu_uc_rtrans, they contain the exact substrings "peripheral_" and
"cpu_uc_". It looks like they might be incorrectly filtered out here, making
them unavailable in sysfs.
> + strstr(attr->name, "cpu_l2_") ||
> + strstr(attr->name, "l2_control_") ||
> + strstr(attr->name, "system_control_") ||
> + strstr(attr->name, "direct_control_") ||
> + strstr(attr->name, "direct_data_"))
> + return 0;
> + } else {
[ ... ]
> @@ -1507,15 +1831,17 @@ static void rpi_axi_pmu_del(struct perf_event *event, int flags)
> if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
> pmu->monitor[mon].hw_events.refcount[idx]--;
> if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
> - if (mon == MON__SYSTEM)
> - rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
> pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
> - pmu->monitor[mon].hw_events.enabled[idx] = false;
> pmu->monitor[mon].hw_events.num_monitored--;
[Severity: High]
Could removing this synchronous clear of enabled[idx] cause a race condition
leading to silent data corruption?
If a VPU monitor event is deleted here, refcount[idx] drops to 0,
monitored_bus[idx] becomes -1 (freeing the slot), and we schedule
pmu->vpu_work.
If a new VPU event is added before rpi_axi_pmu_vpu_work_handler() runs,
it will reuse this empty slot and increment refcount[idx] to 1.
Then when rpi_axi_pmu_vpu_work_handler() finally runs:
drivers/perf/rpi_axi_pmu.c:rpi_axi_pmu_vpu_work_handler() {
...
/* If VPU bus watcher is not enabled on hardware, enable it... */
if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
...
}
...
}
It appears the worker will skip configuring the new hardware because
enabled[idx] remained true from the old event.
Furthermore, the teardown of the old hardware is also skipped later in the
handler:
drivers/perf/rpi_axi_pmu.c:rpi_axi_pmu_vpu_work_handler() {
...
for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
if (pmu->monitor[MON__VPU].hw_events.refcount[idx] == 0 &&
pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
...
}
}
}
Since the new event already incremented refcount[idx], this block won't
execute, potentially leaving the hardware misconfigured and causing userspace
to receive PMU counts for a completely different AXI bus.
> - if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
> - set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> - pmu->monitor[mon].hw_events.monitor_running = false;
> + if (mon == MON__SYSTEM) {
> + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> + if (pmu->monitor[mon].hw_events.num_monitored == 0) {
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> + pmu->monitor[mon].hw_events.monitor_running = false;
> + }
> + } else if (mon == MON__VPU) {
> + schedule_work(&pmu->vpu_work);
> }
> }
> }
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812230800.2646749-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 23:07 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-12 23:25 ` sashiko-bot
2026-08-13 0:06 ` Florian Fainelli
1 sibling, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-12 23:25 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [Critical] rpi_axi_pmu_del() calls hrtimer_cancel() while holding the perf core's ctx->lock (a raw spinlock) and with IRQs disabled, leading to deadlocks.
- [High] VPU hardware bus watchers are never disabled when their associated perf events are deleted, resulting in a permanent hardware resource leak.
- [Medium] Final VPU event counts are lost when an event is stopped, leading to silent under-reporting.
- [Low] RASPBERRYPI_AXI_PMU conditionally selects RASPBERRYPI_FIRMWARE without ensuring its dependencies are met, causing Kconfig warnings.
--
--- Patch [1]: [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit cadc1bb2e8b72c5287e86258dff1cee13436365d
Author: Ian Rogers <irogers@google.com>
Date: Wed Aug 12 23:07:59 2026 +0000
perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
> diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
> index 245e7bb763b99..97c8bf34f4633 100644
> --- a/drivers/perf/Kconfig
> +++ b/drivers/perf/Kconfig
> @@ -325,4 +325,14 @@ config NVIDIA_TEGRA410_C2C_PMU
> Enable perf support for counters in NVIDIA C2C interface of NVIDIA
> Tegra410 SoC.
>
> +config RASPBERRYPI_AXI_PMU
> + tristate "Raspberry Pi AXI PMU support"
> + depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
> + select RASPBERRYPI_FIRMWARE if ARCH_BCM2835
[Severity: Low]
Does this conditionally select RASPBERRYPI_FIRMWARE without ensuring its
dependencies are met? If a user configures ARCH_BCM2835=y but MAILBOX=n,
could this cause Kconfig unmet dependency warnings and silent build failures?
[ ... ]
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..e5cc86fc78b25
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +static void rpi_axi_pmu_read(struct perf_event *event)
> +{
> + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> + enum monitor mon = config_to_monitor(event->attr.config);
> + enum counter counter = config_to_counter(event->attr.config);
> + u64 prev_count, new_count;
> + unsigned long flags;
> + u32 delta;
> +
> + /* Mailbox VPU counters are polled asynchronously in background vpu_work.
> + * MMIO monitors (System and BCM2712 VPU) are read synchronously.
> + */
> + if (pmu->monitor[mon].use_mailbox_interface || event->hw.idx < 0)
> + return;
[Severity: Medium]
When a VPU event is stopped, it looks like rpi_axi_pmu_read() bails out here
without capturing the final delta.
Additionally, in rpi_axi_pmu_vpu_work_handler():
if (!event || (event->hw.state & PERF_HES_STOPPED) ||
config_to_monitor(event->attr.config) != MON__VPU)
continue;
The background poller skips stopped events. If rpi_axi_pmu_stop() marks the
event as stopped, does this result in the final VPU counts being completely
discarded? Could short-lived processes consistently report zero events because
of this?
[ ... ]
> +static void rpi_axi_pmu_del(struct perf_event *event, int flags)
> +{
> + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> + enum monitor mon = config_to_monitor(event->attr.config);
> + unsigned long spinflags;
> + int idx = event->hw.idx;
> +
> + if (idx < 0)
> + return;
> +
> + rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
> +
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
> + if (pmu->events[i] == event) {
> + pmu->events[i] = NULL;
> + pmu->event_gen[i]++;
> + break;
> + }
> + }
> +
> + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
> + pmu->monitor[mon].hw_events.refcount[idx]--;
> + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
> + if (mon == MON__SYSTEM)
> + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
> + pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
> + pmu->monitor[mon].hw_events.enabled[idx] = false;
[Severity: High]
Does this cause a permanent hardware resource leak for VPU events? Since
rpi_axi_pmu_disable_bus_watcher() is only called for MON__SYSTEM, the VPU
bus watchers are never disabled. Furthermore, because enabled[idx] is set to
false, the hardware is left running indefinitely with no way for any background
polling logic to track or disable it later.
> + pmu->monitor[mon].hw_events.num_monitored--;
> + if (mon == MON__SYSTEM && pmu->monitor[mon].hw_events.num_monitored == 0) {
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> + pmu->monitor[mon].hw_events.monitor_running = false;
> + }
> + }
> + }
> +
> + event->hw.idx = -1;
> + pmu->active_events--;
> + if (mon == MON__VPU)
> + pmu->active_vpu_events--;
> +
> + if (pmu->active_events == 0) {
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> + if (in_hardirq())
> + hrtimer_try_to_cancel(&pmu->hrtimer);
> + else
> + hrtimer_cancel(&pmu->hrtimer);
[Severity: Critical]
Can this lead to deadlocks or scheduling while atomic panics?
rpi_axi_pmu_del() is called with the perf core's ctx->lock held (a raw spinlock)
and local IRQs disabled. If in_hardirq() is false, this branch calls
hrtimer_cancel().
Because this is a SOFT hrtimer (HRTIMER_MODE_REL_SOFT), hrtimer_cancel()
can sleep on PREEMPT_RT, causing an immediate panic. Even without PREEMPT_RT,
if the user thread preempted ksoftirqd on the same CPU, spin-waiting for
the timer to finish while preemption is disabled causes a permanent
self-deadlock.
> + spin_lock_irqsave(&pmu->lock, spinflags);
> + if (pmu->active_events > 0)
> + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
> + HRTIMER_MODE_REL_SOFT);
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> + return;
> + }
> + spin_unlock_irqrestore(&pmu->lock, spinflags);
> +}
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260812230800.2646749-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-12 23:07 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 23:25 ` sashiko-bot
@ 2026-08-13 0:06 ` Florian Fainelli
1 sibling, 0 replies; 59+ messages in thread
From: Florian Fainelli @ 2026-08-13 0:06 UTC (permalink / raw)
To: Ian Rogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
On 8/12/26 16:07, Ian Rogers wrote:
> This commit adds an uncore performance monitoring unit (PMU) driver
> for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
> mailbox monitors found on Raspberry Pi 1 through 4.
Missing generative AI tags here?
>
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
> drivers/perf/Kconfig | 10 +
> drivers/perf/Makefile | 1 +
> drivers/perf/rpi_axi_pmu.c | 1793 ++++++++++++++++++++++++++++++++++++
> 3 files changed, 1804 insertions(+)
> create mode 100644 drivers/perf/rpi_axi_pmu.c
This is more applicable to the second part of the patch, but the
dependency on the Raspberry Pi firmware interface bothers me a bit. We
have the AXI PMU blocks available on the 7712/77122 chips which are
sister chips to the 2712, we don't run the Raspberry Pi firmware on
those systems however.
Once your patches land, I will see if I can get generative AI to help me
add support for the two AXI PMUs that exist on those chips.
Thanks for doing this!
--
Florian
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
` (3 preceding siblings ...)
2026-08-12 23:07 ` Ian Rogers
@ 2026-08-13 4:35 ` Ian Rogers
2026-08-13 4:35 ` [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
4 siblings, 3 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 4:35 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
------------------------
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
------------------------------------------
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq context,
ensuring spinlock acquisitions are 100% PREEMPT_RT safe.
- Sets suppress_bind_attrs = true to prevent unsafe manual sysfs unbinding while
active perf events exist.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
-------------------
The driver has been rigorously validated on physical hardware across multiple
SoC generations:
- Raspberry Pi 400 (BCM2711): Validated System MMIO and VideoCore VPU firmware
mailbox IPC performance counters. Explicitly mechanically verified precisely zero
lockup behaviors across 100Hz aggressive asynchronous timer interrupts. Profiling
the `h264_v4l2m2m` pipeline hardware isolated >11.1MB in pure AXI VPU hardware matrix reads.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers.
Changes in v6
-------------
- Reorganized commit structure rigorously into two cleanly bisectable patches
(BCM2835 core in Patch 1, BCM2712 additions in Patch 2).
- Resolved unmet Kconfig dependency warnings when MAILBOX=n by cleanly relying
on 'depends on RASPBERRYPI_FIRMWARE || (COMPILE_TEST && !RASPBERRYPI_FIRMWARE)'
rather than forcefully using 'select' (Patch 1).
- Resolved catastrophic PREEMPT_RT/softirq deadlocks associated with atomic
spinlocks globally preempting 'hrtimer_cancel()' routines by permitting the
timer loop to detect 0 active events and cleanly self-terminate (Patch 1).
- Purged unsafe 'spin_unlock_irqrestore' lock dropping maneuvers occurring
abruptly inside 'rpi_axi_pmu_read()', ensuring atomic contexts and missing
flags no longer inadvertently trigger kernel panics (Patch 1).
- Eliminated silent ABA VPU slot data-corruption and event hardware leaks by
decoupling 'vpu_work_handler' loops safely onto a 'vpu_disable_pending' matrix.
Deleted VPU watchers are safely disabled exactly prior to newly allocated slots
executing initialization within the same asynchronous worker loop (Patch 1).
- Resolved lockless data races across VPU mailbox hardware mutation checks by fully
protecting 'monitor_running' and 'enabled' state modifications structurally inside
the 'rpi_axi_pmu_start' and 'rpi_axi_pmu_vpu_work_handler' core locks.
- Disentangled rigid 'cancel_work_sync' usages back to safe 'flush_work' variants in
teardown unloading contexts so dynamic Mailbox IPC teardown flows successfully
execute without aborting directly in mid-flight (Patch 2).
- Sharpened the sysfs 'is_visible' string-masking logic to properly accommodate
Raspberry Pi 5 aliases, resolving an overzealous 'strstr' bug directly hiding
"bcm2712_peripheral_" aliases on RPi 5 hardware (Patch 2).
- Rewrote component destruction sequencing logic completely onto safe CPU hotplug
'_nocalls' handlers to strictly avoid User-After-Free events from persisting
against unregistered/defunct driver models during 'perf_pmu_unregister' (Patch 1).
- Eradicated cross-execution bus tracker leaks across failed/zombied workflows where
'monitored' mask properties remained structurally trapped inside invalid state
loops by surgically injecting manual 'enabled = false' boolean teardowns into 'del()'.
- Inserted explicit comments explaining the critical 31-bit counter-masking logic
regarding standard U32_MAX hardware exception signaling (Patch 1).
- Scrubbed all erroneous 'TAG=agy' trailing metadata from commit descriptions.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 9 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2119 ++++++++++++++++++++++++++++++++++++
3 files changed, 2129 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.47.0
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 4:35 ` [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-13 4:35 ` Ian Rogers
2026-08-13 4:48 ` sashiko-bot
2026-08-13 4:35 ` [PATCH v6 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 5:24 ` [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 4:35 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/Kconfig | 9 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2009 ++++++++++++++++++++++++++++++++++++
3 files changed, 2019 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..254293bcb3c5 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -325,4 +325,13 @@ config NVIDIA_TEGRA410_C2C_PMU
Enable perf support for counters in NVIDIA C2C interface of NVIDIA
Tegra410 SoC.
+config RASPBERRYPI_AXI_PMU
+ tristate "Raspberry Pi AXI PMU support"
+ depends on ARCH_BRCMSTB || RASPBERRYPI_FIRMWARE || (COMPILE_TEST && !RASPBERRYPI_FIRMWARE)
+ help
+ Support for the AXI Performance Monitoring Unit (PMU) found on Broadcom
+ SoCs used in Raspberry Pi platforms (BCM2835, BCM2836, BCM2837, BCM2711, BCM2712).
+
+ If compiled as a module, it will be called rpi_axi_pmu.
+
endmenu
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..29d3915d2dbe 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/
obj-$(CONFIG_CXL_PMU) += cxl_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o
obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o
+obj-$(CONFIG_RASPBERRYPI_AXI_PMU) += rpi_axi_pmu.o
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..f7e5520d02eb
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,2009 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+bool monitor_running;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+ bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
+ return false;
+ } else {
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+ }
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Bus Events (bus=1, BCM2835_SB__TRANS: format rotation & matrix conversion) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans, "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans, "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans, "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans, "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans, "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans, "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans, "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans, "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans, "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans, "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans, "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans, "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans, "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans, "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans, "monitor=0,bus=5,counter=2,filter=24");
+
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+
+/* BCM2712 Master ID Filtered Events */
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+
+
+
+
+
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
+ */
+ if (event->hw.idx < 0)
+ return;
+
+ if (pmu->monitor[mon].use_mailbox_interface)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx]) {
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = false;
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = true;
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM) {
+ pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] = false;
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = false;
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = true;
+ schedule_work(&pmu->vpu_work);
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp_remove;
+ }
+
+ return 0;
+
+err_cpuhp_remove:
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ flush_work(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ perf_pmu_unregister(&pmu->pmu);
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ flush_work(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v6 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 4:35 ` [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 4:35 ` [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 4:35 ` Ian Rogers
2026-08-13 4:43 ` sashiko-bot
2026-08-13 5:24 ` [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 4:35 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
(Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
and Cortex-A76 DSU L3 interconnects.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 120 +++++++++++++++++++++++++++++++++++--
1 file changed, 115 insertions(+), 5 deletions(-)
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index f7e5520d02eb..4c1b19c6cc2e 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -867,19 +867,45 @@ PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtra
/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, "monitor=0,bus=6,counter=5");
/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, "monitor=0,bus=7,counter=4");
/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans, "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans, "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans, "monitor=0,bus=8,counter=4");
/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, "monitor=0,bus=10,counter=4");
/*
* Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
* (bus=14, BCM2712_SB__CPU_L2)
*/
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans, "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans, "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans, "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans, "monitor=0,bus=13,counter=2");
/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans, "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, "monitor=0,bus=5,counter=2,filter=29");
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
@@ -1039,11 +1065,37 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
/* RPi 5 (BCM2712) Expanded Event Aliases */
-
-
-
-
-
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
@@ -1059,10 +1111,64 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
*
* Return: attr->mode (0444) if visible on current SoC, 0 to hide.
*/
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ (strstr(attr->name, "peripheral_") && !strstr(attr->name, "bcm2712_")) ||
+ (strstr(attr->name, "cpu_uc_") && !strstr(attr->name, "bcm2712_")) ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_") ||
+ strstr(attr->name, "bcm2712_"))
+ return 0;
+ }
+
+ return attr->mode;
+}
static const struct attribute_group rpi_axi_pmu_events_group = {
.name = "events",
.attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
};
/**
@@ -1954,6 +2060,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v6 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 4:35 ` [PATCH v6 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-13 4:43 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 4:43 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
> Expand the Raspberry Pi AXI PMU driver to support Broadcom BCM2712
> (Raspberry Pi 5) system monitors, PCIe RP1 Southbridge links, HEVC decoder,
> and Cortex-A76 DSU L3 interconnects.
>
> Signed-off-by: Ian Rogers <irogers@google.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813043533.2744442-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 4:35 ` [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 4:48 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 4:48 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] Concurrent execution of `rpi_axi_pmu_del()`, `rpi_axi_pmu_add()`, and the background workqueue can leave a VPU hardware bus watcher permanently disabled due to unprotected mutation of the `vpu_disable_pending` flag.
- [Low] The BCM2712 (Raspberry Pi 5) specific event aliases referenced in the `rpi_axi_pmu_events_attrs` array are missing their macro definitions, causing compilation to fail.
- [Low] Usage of LINUX_VERSION_CODE check for out-of-tree backward compatibility in a mainline patch.
--
--- Patch [1]: [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit cb96f82b55ab634031075f8a381cb2d336d091c7
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds an uncore performance monitoring unit (PMU) driver
for Broadcom BCM2835-BCM2711 AXI system monitors and VideoCore VPU firmware
mailbox monitors found on Raspberry Pi 1 through 4.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..f7e5520d02eb9
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
> @@ -0,0 +1,2009 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +
> +/**
> + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
> + *
> + * This driver exposes the performance monitoring hardware on Raspberry Pi
[ ... ]
> +#include <linux/version.h>
> +
> +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
> +static inline void rpi_hrtimer_setup(struct hrtimer *timer,
> + enum hrtimer_restart (*function)(struct hrtimer *),
> + clockid_t clock_id, enum hrtimer_mode mode)
> +{
[Severity: Low]
Should this out-of-tree backward compatibility check using LINUX_VERSION_CODE
be included in a mainline patch? Typically, mainline submissions only target
the current kernel version and avoid legacy version checks.
> + hrtimer_init(timer, clock_id, mode);
> + timer->function = function;
> +}
> +#define hrtimer_setup rpi_hrtimer_setup
> +#endif
[ ... ]
> +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
> + const struct perf_event *event)
> +{
> + int bus = config_to_bus(event->attr.config);
> + int filter = config_to_filter(event->attr.config);
> +
> + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
> + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
> + hw_events->refcount[i]++;
> + return i;
> + }
> + }
> + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
> + return -1;
> +
> + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
> + if (hw_events->monitored_bus[i] == -1) {
> + hw_events->monitored_bus[i] = bus;
> + hw_events->filter[i] = filter;
> + hw_events->refcount[i] = 1;
[Severity: Medium]
Can a newly allocated VPU watcher be immediately and permanently disabled due
to a stale vpu_disable_pending flag?
If a concurrent rpi_axi_pmu_del() frees a watcher and sets:
vpu_disable_pending[idx] = true;
enabled[idx] = false;
schedule_work(&pmu->vpu_work);
And then rpi_axi_hw_events__get_alloc_event_idx() reuses this idx for a new
event before the workqueue runs, it does not clear vpu_disable_pending here.
When rpi_axi_pmu_vpu_work_handler() later processes the new event, it will
enable it in hardware and set enabled[idx] = true. However, on the next run,
the handler will see the stale vpu_disable_pending == true, disable the
hardware, but skip re-enabling it because enabled[idx] is already true,
leaving the hardware counter silently stopped.
> + hw_events->num_monitored++;
> + return i;
> + }
> + }
> + return -1;
> +}
[ ... ]
> + /* Filtered Event Aliases (RPi 1-4) */
> + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
> + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
> + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
> + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
> + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
> + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
> + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
> + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
> + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
> + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
> + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
> + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
> + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
> + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
> +
> + /* RPi 5 (BCM2712) Expanded Event Aliases */
> + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
> + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
[Severity: Low]
Are these BCM2712 event aliases missing their corresponding macro definitions?
It looks like the array references variables like
rpi_axi_pmu_event_pcie_rp1_atrans that were never declared with
PMU_EVENT_ATTR_STRING, which would cause a compiler error for undefined
identifiers when building the driver.
> + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
> + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
> + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
> + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
> +
> + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
[ ... ]
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813043533.2744442-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-13 4:35 ` [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 4:35 ` [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-13 4:35 ` [PATCH v6 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-13 5:24 ` Ian Rogers
2026-08-13 5:24 ` [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
2 siblings, 3 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 5:24 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
=======================
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling.
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
=======================
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via sysfs boundaries.
- Supports event sampling and hardware counter accumulation,
automatically managing 31-bit hardware counter wraparound.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context to an online CPU core when a
designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads
directly mapped over ARM physical memory space.
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms,
VideoCore VPU monitor IPC calls are offloaded to process context via
a dedicated workqueue (`vpu_work`) and serialized.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq
context, ensuring spinlock acquisitions are PREEMPT_RT safe.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711.
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5).
Changes in v7
=======================
- Mathematically scrubbed Patch 1 to explicitly isolate ALL 'bcm2712'
constants, comments, and logic strictly into Patch 2.
- Explicitly initialized 'vpu_disable_pending' to false during bus
watcher allocation to prevent orphaned hardware disables from stale
recycled pointers (Patch 1).
- Maintained the out-of-tree LTS backward compatibility block exactly
to preserve simultaneous compilation against Raspberry Pi Foundation
downstream kernels.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/rpi_axi_pmu.c | 2285 ++++++++++++++++++++++++++++++++++++
1 file changed, 2285 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.47.0
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 5:24 ` [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-13 5:24 ` Ian Rogers
2026-08-13 5:35 ` sashiko-bot
2026-08-13 5:24 ` [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 6:42 ` [PATCH v8 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 5:24 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 1993 ++++++++++++++++++++++++++++++++++++
1 file changed, 1993 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..bd3c79dd2ff8
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,1993 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ * @monitor_running: Flag indicating if the hardware monitor loop is globally active
+ * @vpu_disable_pending: Array tracking asynchronous hardware disable requests for VPU
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+bool monitor_running;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+ bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->vpu_disable_pending[i] = false;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans,
+ "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait,
+ "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans,
+ "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait,
+ "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans,
+ "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait,
+ "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Events (bus=1, BCM2835_SB__TRANS: format rotation) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans,
+ "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait,
+ "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans,
+ "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait,
+ "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans,
+ "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait,
+ "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans,
+ "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait,
+ "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans,
+ "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait,
+ "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans,
+ "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait,
+ "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans,
+ "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait,
+ "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans,
+ "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait,
+ "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans,
+ "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait,
+ "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans,
+ "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait,
+ "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans,
+ "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait,
+ "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans,
+ "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait,
+ "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans,
+ "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait,
+ "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait,
+ "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait,
+ "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans,
+ "monitor=0,bus=8,counter=0");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans,
+ "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait,
+ "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans,
+ "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait,
+ "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans,
+ "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait,
+ "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait,
+ "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait,
+ "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait,
+ "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans,
+ "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait,
+ "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans,
+ "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait,
+ "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans,
+ "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait,
+ "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans,
+ "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait,
+ "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait,
+ "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait,
+ "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans,
+ "monitor=0,bus=13,counter=0");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans,
+ "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait,
+ "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans,
+ "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait,
+ "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans,
+ "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait,
+ "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans,
+ "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait,
+ "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans,
+ "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait,
+ "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans,
+ "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait,
+ "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans,
+ "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans,
+ "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans,
+ "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans,
+ "monitor=1,bus=3,counter=0");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans,
+ "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans,
+ "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans,
+ "monitor=1,bus=4,counter=4");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans,
+ "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans,
+ "monitor=1,bus=6,counter=0");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans,
+ "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans,
+ "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans,
+ "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans,
+ "monitor=1,bus=10,counter=0");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans,
+ "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans,
+ "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans,
+ "monitor=1,bus=11,counter=4");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans,
+ "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans,
+ "monitor=1,bus=13,counter=0");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans,
+ "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait,
+ "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans,
+ "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait,
+ "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans,
+ "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait,
+ "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans,
+ "monitor=1,bus=15,counter=0");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=6");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=17");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=26");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=24");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ return attr->mode;
+}
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System) are read synchronously.
+ */
+ if (event->hw.idx < 0)
+ return;
+
+ if (pmu->monitor[mon].use_mailbox_interface)
+ return;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ spin_lock_irq(&pmu->lock);
+
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx]) {
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = false;
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = true;
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
+ spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM) {
+ pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] = false;
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = false;
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = true;
+ schedule_work(&pmu->vpu_work);
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp_remove;
+ }
+
+ return 0;
+
+err_cpuhp_remove:
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ flush_work(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ perf_pmu_unregister(&pmu->pmu);
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ hrtimer_cancel(&pmu->hrtimer);
+ flush_work(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 5:24 ` [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 5:24 ` [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 5:24 ` Ian Rogers
2026-08-13 5:46 ` sashiko-bot
2026-08-13 6:42 ` [PATCH v8 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 5:24 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 308 ++++++++++++++++++++++++++++++++++++-
1 file changed, 300 insertions(+), 8 deletions(-)
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index bd3c79dd2ff8..a843e8636d81 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,6 +6,7 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,43 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
@@ -259,6 +300,93 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
@@ -435,7 +563,7 @@ static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_e
* struct rpi_axi_pmu - Root PMU driver context
* @pmu: Core Linux perf PMU structure
* @pdev: Owning platform_device pointer
- * @chip: Detected Broadcom SoC generation
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
* @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
* @cpu: CPU core assigned to process uncore PMU events
* @cpuhp_node: Dynamic CPU hotplug instance node
@@ -495,15 +623,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
return counter < CNT__MAX;
}
@@ -529,7 +666,7 @@ static const struct attribute_group rpi_axi_pmu_format_group = {
* Event Attribute Scaling & Unit Definitions:
*
* Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
- * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
* transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
* use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
* close (~95%) byte throughput approximation in perf stat output (accounting for
@@ -867,6 +1004,75 @@ PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtra
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans,
"monitor=0,bus=5,counter=2,filter=24");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait,
+ "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans,
+ "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait,
+ "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans,
+ "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait,
+ "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans,
+ "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans,
+ "monitor=0,bus=7,counter=4");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans,
+ "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans,
+ "monitor=0,bus=8,counter=4");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans,
+ "monitor=0,bus=10,counter=4");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans,
+ "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans,
+ "monitor=0,bus=13,counter=2");
+
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans,
+ "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans,
+ "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans,
+ "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans,
+ "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans,
+ "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans,
+ "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans,
+ "monitor=0,bus=5,counter=2,filter=29");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -1024,6 +1230,38 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
NULL,
};
@@ -1034,12 +1272,62 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
* @unused: Index (unused)
*
* Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
*
* Return: attr->mode (0444) if visible on current SoC, 0 to hide.
*/
static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
struct attribute *attr, int unused)
{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ (strstr(attr->name, "peripheral_") && !strstr(attr->name, "bcm2712_")) ||
+ (strstr(attr->name, "cpu_uc_") && !strstr(attr->name, "bcm2712_")) ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_") ||
+ strstr(attr->name, "bcm2712_"))
+ return 0;
+ }
+
return attr->mode;
}
@@ -1370,7 +1658,7 @@ static void rpi_axi_pmu_read(struct perf_event *event)
u32 delta;
/* Mailbox VPU counters are polled asynchronously in background vpu_work.
- * MMIO monitors (System) are read synchronously.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
*/
if (event->hw.idx < 0)
return;
@@ -1938,6 +2226,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 5:24 ` [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 5:35 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 5:35 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] VPU monitor accounting is broken, causing complete data loss during time-multiplexing and short profiling sessions.
- [High] The driver uses `spinlock_t` for `pmu->lock`, which crashes PREEMPT_RT kernels with "sleeping while atomic".
--
--- Patch [1]: [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit a038f5a0d4388926d4738fa191fa39c1cd2e9f2a
Author: Ian Rogers <irogers@google.com>
Subject: perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds a performance monitoring driver for the Raspberry Pi AXI
bus. It supports both System and VPU monitors.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> index 0000000000000..bd3c79dd2ff8d
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +struct rpi_axi_pmu {
> + struct pmu pmu;
> + struct platform_device *pdev;
> + enum rpi_axi_chip chip;
> + struct rpi_firmware *firmware;
> +
> + int cpu;
> + struct hlist_node cpuhp_node;
> +
> + spinlock_t lock;
[Severity: High]
Will using a standard spinlock_t here cause crashes on PREEMPT_RT kernels?
The perf core invokes PMU callbacks (like add, del, start, stop, and read)
with hard IRQs disabled. On PREEMPT_RT, spinlock_t is mapped to a sleepable
rt_mutex.
Acquiring pmu->lock via spin_lock_irqsave() in these atomic paths, such as
in rpi_axi_pmu_add():
rpi_axi_pmu_add() {
...
spin_lock_irqsave(&pmu->lock, spinflags);
...
}
will trigger a "sleeping while atomic" crash. Should this be changed to a
raw_spinlock_t to guarantee it does not sleep?
> + struct mutex vpu_mutex;
[ ... ]
> +static void rpi_axi_pmu_read(struct perf_event *event)
> +{
> + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> + enum monitor mon = config_to_monitor(event->attr.config);
> + enum counter counter = config_to_counter(event->attr.config);
> + u64 prev_count, new_count;
> + unsigned long flags;
> + u32 delta;
> +
> + /* Mailbox VPU counters are polled asynchronously in background vpu_work.
> + * MMIO monitors (System) are read synchronously.
> + */
> + if (event->hw.idx < 0)
> + return;
> +
> + if (pmu->monitor[mon].use_mailbox_interface)
> + return;
[Severity: High]
Does returning immediately here lead to complete data loss for VPU events
during time-multiplexing and short profiling sessions?
When the perf core stops an event (for instance, during event rotation or at
the end of a session), it calls rpi_axi_pmu_read() to capture the final count.
Because VPU events bypass synchronous reading here, any counts accumulated
since the last 2-second background timer tick are not recorded.
When the event is subsequently restarted, the background work sets a new
baseline:
rpi_axi_pmu_vpu_work_handler() {
...
if (!(event->hw.state & PERF_HES_UPTODATE)) {
/* Initial baseline read for newly started/rotated VPU event */
local64_set(&event->hw.prev_count, new_count);
event->hw.state |= PERF_HES_UPTODATE;
}
...
}
This permanently strands the uncollected counts from the previous run period.
Could this cause short-lived perf stat sessions (under 2 seconds) and
oversubscribed VPU events to systematically report near zero?
> +
> + spin_lock_irqsave(&pmu->lock, flags);
[ ... ]
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813052419.2770841-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 5:24 ` [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-13 5:46 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 5:46 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
> Signed-off-by: Ian Rogers <irogers@google.com>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813052419.2770841-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v8 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-13 5:24 ` [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 5:24 ` [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-13 5:24 ` [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-13 6:42 ` Ian Rogers
2026-08-13 6:42 ` [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
` (2 more replies)
2 siblings, 3 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 6:42 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
=======================
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling.
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
=========================================
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via sysfs boundaries.
- Supports event sampling and hardware counter accumulation,
automatically managing 31-bit hardware counter wraparound.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context to an online CPU core when a
designated CPU goes offline, avoiding stale uncore state.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads
directly mapped over ARM physical memory space.
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms,
VideoCore VPU monitor IPC calls are offloaded to process context via
a dedicated workqueue (`vpu_work`) and serialized.
4. PREEMPT_RT & Safety Hardening:
- Uses HRTIMER_MODE_REL_SOFT for timer callbacks to execute in softirq
context, avoiding local hard interrupt nesting priority deadlocks.
5. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711.
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5).
Changes in v8
=============
- Fully transitioned locking architecture from 'spinlock_t' globally to
'raw_spinlock_t' to mathematically guarantee fastpath safety against
"scheduling while atomic" crashes during PREEMPT_RT deployment where
the perf core unconditionally disables hard IRQs prior to PMU callbacks.
- Re-architected 'RPI_AXI_PMU_TIMER_INTERVAL' significantly downward from
2000ms natively to 100ms. This acts as a structural baseline compromise
that prevents excessive VideoCore Mailbox IPC context-switch wakeovers
on older Pi 1/2 boards, while safely minimizing uncollected residuals
causing 0-byte dropouts during time multiplexing/undersubscribed traces.
- Enforced strict 100-column sysfs string macro breakages globally.
- Added proper .scale and .unit bytes aliases across all AXI transaction
events.
- Fixed a fatal use-after-free initialization race condition catching cases
where cpuhp_state instances were registered prior to the PMU backend,
which allowed concurrent CPU hotplug migrations to panic aggressively
against unregistered core structures.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/rpi_axi_pmu.c | 2884 ++++++++++++++++++++++++++++++++++++
1 file changed, 2884 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.47.0
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 6:42 ` [PATCH v8 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
@ 2026-08-13 6:42 ` Ian Rogers
2026-08-13 6:51 ` sashiko-bot
2026-08-13 6:42 ` [PATCH v8 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 6:42 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 2450 ++++++++++++++++++++++++++++++++++++
1 file changed, 2450 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..676c2a3d5962
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,2450 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+/*
+ * RPI_AXI_PMU_TIMER_INTERVAL determines the background polling frequency
+ * for VideoCore VPU Mailbox IPC counters.
+ *
+ * A balance is required:
+ * - IPC Overhead: Polling overly fast (e.g., 10ms) generates excessive CPU
+ * wakeups and VideoCore IPC interrupts on older CPUs (Pi 1/2).
+ * - Accuracy: Polling overly slow (e.g., 2000ms) causes short time-multiplexed
+ * profiling sessions (under the interval) to mathematically strand residual
+ * counts since the mailbox cannot be queried synchronously inside pmu->read().
+ *
+ * 100ms (10 Hz) provides reasonably accurate profiling without heavy overhead.
+ */
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(100)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ * @monitor_running: Flag indicating if the hardware monitor loop is globally active
+ * @vpu_disable_pending: Array tracking asynchronous hardware disable requests for VPU
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+bool monitor_running;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+ bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->vpu_disable_pending[i] = false;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+
+ raw_spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans,
+ "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atrans.scale, rpi_axi_pmu_event_dma_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_atrans.unit, rpi_axi_pmu_event_dma_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait,
+ "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans,
+ "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait,
+ "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans,
+ "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait,
+ "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Events (bus=1, BCM2835_SB__TRANS: format rotation) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans,
+ "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atrans.scale, rpi_axi_pmu_event_trans_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_atrans.unit, rpi_axi_pmu_event_trans_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait,
+ "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans,
+ "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtrans.scale, rpi_axi_pmu_event_trans_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_wtrans.unit, rpi_axi_pmu_event_trans_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait,
+ "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans,
+ "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtrans.scale, rpi_axi_pmu_event_trans_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_rtrans.unit, rpi_axi_pmu_event_trans_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait,
+ "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans,
+ "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atrans.scale, rpi_axi_pmu_event_jpeg_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_atrans.unit, rpi_axi_pmu_event_jpeg_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait,
+ "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans,
+ "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans.scale, rpi_axi_pmu_event_jpeg_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans.unit, rpi_axi_pmu_event_jpeg_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait,
+ "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans,
+ "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans.scale, rpi_axi_pmu_event_jpeg_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans.unit, rpi_axi_pmu_event_jpeg_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait,
+ "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans,
+ "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atrans.scale, rpi_axi_pmu_event_system_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_atrans.unit, rpi_axi_pmu_event_system_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait,
+ "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans,
+ "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans.scale, rpi_axi_pmu_event_system_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans.unit, rpi_axi_pmu_event_system_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait,
+ "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans,
+ "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans.scale, rpi_axi_pmu_event_system_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans.unit, rpi_axi_pmu_event_system_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait,
+ "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans,
+ "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atrans.scale, rpi_axi_pmu_event_dma_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_atrans.unit, rpi_axi_pmu_event_dma_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait,
+ "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans,
+ "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans.scale, rpi_axi_pmu_event_dma_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans.unit, rpi_axi_pmu_event_dma_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait,
+ "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans,
+ "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans.scale, rpi_axi_pmu_event_dma_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans.unit, rpi_axi_pmu_event_dma_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait,
+ "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans,
+ "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atrans.scale, rpi_axi_pmu_event_system_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_atrans.unit, rpi_axi_pmu_event_system_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait,
+ "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait,
+ "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait,
+ "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans.scale, rpi_axi_pmu_event_ccp2tx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans.unit, rpi_axi_pmu_event_ccp2tx_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans.scale, rpi_axi_pmu_event_mphi_rx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans.unit, rpi_axi_pmu_event_mphi_rx_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans.scale, rpi_axi_pmu_event_mphi_tx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans.unit, rpi_axi_pmu_event_mphi_tx_atrans_unit,
+ "Bytes");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans,
+ "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atrans.scale, rpi_axi_pmu_event_hvs_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_atrans.unit, rpi_axi_pmu_event_hvs_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait,
+ "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans,
+ "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait,
+ "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans,
+ "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait,
+ "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atrans.scale, rpi_axi_pmu_event_h264_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_atrans.unit, rpi_axi_pmu_event_h264_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait,
+ "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtrans.scale, rpi_axi_pmu_event_h264_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_wtrans.unit, rpi_axi_pmu_event_h264_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait,
+ "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtrans.scale, rpi_axi_pmu_event_h264_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_rtrans.unit, rpi_axi_pmu_event_h264_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait,
+ "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans,
+ "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atrans.scale, rpi_axi_pmu_event_isp_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_atrans.unit, rpi_axi_pmu_event_isp_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait,
+ "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans,
+ "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtrans.scale, rpi_axi_pmu_event_isp_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_wtrans.unit, rpi_axi_pmu_event_isp_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait,
+ "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans,
+ "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtrans.scale, rpi_axi_pmu_event_isp_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_rtrans.unit, rpi_axi_pmu_event_isp_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait,
+ "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans,
+ "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atrans.scale, rpi_axi_pmu_event_v3d_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_atrans.unit, rpi_axi_pmu_event_v3d_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait,
+ "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait,
+ "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait,
+ "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans,
+ "monitor=0,bus=13,counter=0");
+PMU_EVENT_ATTR_STRING(peripheral_atrans.scale, rpi_axi_pmu_event_peripheral_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(peripheral_atrans.unit, rpi_axi_pmu_event_peripheral_atrans_unit,
+ "Bytes");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans,
+ "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans.scale, rpi_axi_pmu_event_cpu_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans.unit, rpi_axi_pmu_event_cpu_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait,
+ "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans,
+ "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans.scale, rpi_axi_pmu_event_cpu_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans.unit, rpi_axi_pmu_event_cpu_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait,
+ "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans,
+ "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans.scale, rpi_axi_pmu_event_cpu_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans.unit, rpi_axi_pmu_event_cpu_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait,
+ "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans,
+ "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans.scale, rpi_axi_pmu_event_cpu_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans.unit, rpi_axi_pmu_event_cpu_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait,
+ "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans,
+ "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans.scale, rpi_axi_pmu_event_cpu_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans.unit, rpi_axi_pmu_event_cpu_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait,
+ "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans,
+ "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans.scale, rpi_axi_pmu_event_cpu_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans.unit, rpi_axi_pmu_event_cpu_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait,
+ "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans,
+ "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans.scale, rpi_axi_pmu_event_vpu1_d_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans.unit, rpi_axi_pmu_event_vpu1_d_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans,
+ "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans.scale, rpi_axi_pmu_event_vpu0_d_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans.unit, rpi_axi_pmu_event_vpu0_d_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans,
+ "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans.scale, rpi_axi_pmu_event_vpu1_i_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans.unit, rpi_axi_pmu_event_vpu1_i_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans,
+ "monitor=1,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans.scale, rpi_axi_pmu_event_vpu0_i_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans.unit, rpi_axi_pmu_event_vpu0_i_l2_atrans_unit,
+ "Bytes");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans,
+ "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans.scale, rpi_axi_pmu_event_vpu_system_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans.unit, rpi_axi_pmu_event_vpu_system_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans,
+ "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans.scale, rpi_axi_pmu_event_vpu_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans.unit, rpi_axi_pmu_event_vpu_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans,
+ "monitor=1,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans.scale, rpi_axi_pmu_event_vpu_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans.unit, rpi_axi_pmu_event_vpu_system_l2_rtrans_unit,
+ "Bytes");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans,
+ "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans.scale, rpi_axi_pmu_event_vpu_l2_flush_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans.unit, rpi_axi_pmu_event_vpu_l2_flush_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans,
+ "monitor=1,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans.scale, rpi_axi_pmu_event_vpu_dma_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans.unit, rpi_axi_pmu_event_vpu_dma_l2_atrans_unit,
+ "Bytes");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans,
+ "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans.scale, rpi_axi_pmu_event_vpu1_d_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans.unit, rpi_axi_pmu_event_vpu1_d_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans,
+ "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans.scale, rpi_axi_pmu_event_vpu0_d_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans.unit, rpi_axi_pmu_event_vpu0_d_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans,
+ "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans.scale, rpi_axi_pmu_event_vpu1_i_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans.unit, rpi_axi_pmu_event_vpu1_i_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans,
+ "monitor=1,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans.scale, rpi_axi_pmu_event_vpu0_i_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans.unit, rpi_axi_pmu_event_vpu0_i_uc_atrans_unit,
+ "Bytes");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans,
+ "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans.scale, rpi_axi_pmu_event_vpu_system_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans.unit, rpi_axi_pmu_event_vpu_system_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans,
+ "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans.scale, rpi_axi_pmu_event_vpu_system_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans.unit, rpi_axi_pmu_event_vpu_system_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans,
+ "monitor=1,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans.scale, rpi_axi_pmu_event_vpu_system_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans.unit, rpi_axi_pmu_event_vpu_system_uc_rtrans_unit,
+ "Bytes");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans,
+ "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans.scale, rpi_axi_pmu_event_vpu_l2_out_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans.unit, rpi_axi_pmu_event_vpu_l2_out_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans,
+ "monitor=1,bus=13,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans.scale, rpi_axi_pmu_event_vpu_dma_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans.unit, rpi_axi_pmu_event_vpu_dma_uc_atrans_unit,
+ "Bytes");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans,
+ "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans.scale, rpi_axi_pmu_event_vpu_sdram_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans.unit, rpi_axi_pmu_event_vpu_sdram_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait,
+ "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans,
+ "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans.scale, rpi_axi_pmu_event_vpu_sdram_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans.unit, rpi_axi_pmu_event_vpu_sdram_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait,
+ "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans,
+ "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans.scale, rpi_axi_pmu_event_vpu_sdram_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans.unit, rpi_axi_pmu_event_vpu_sdram_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait,
+ "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans,
+ "monitor=1,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans.scale, rpi_axi_pmu_event_vpu_l2_in_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans.unit, rpi_axi_pmu_event_vpu_l2_in_atrans_unit,
+ "Bytes");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans.scale, rpi_axi_pmu_event_cpu0_icache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans.unit, rpi_axi_pmu_event_cpu0_icache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans.scale, rpi_axi_pmu_event_cpu0_dcache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans.unit, rpi_axi_pmu_event_cpu0_dcache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans.scale, rpi_axi_pmu_event_cpu0_dcache_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans.unit, rpi_axi_pmu_event_cpu0_dcache_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans.scale, rpi_axi_pmu_event_cpu1_icache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans.unit, rpi_axi_pmu_event_cpu1_icache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans.scale, rpi_axi_pmu_event_cpu1_dcache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans.unit, rpi_axi_pmu_event_cpu1_dcache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans.scale, rpi_axi_pmu_event_cpu1_dcache_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans.unit, rpi_axi_pmu_event_cpu1_dcache_wtrans_unit,
+ "Bytes");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans.scale, rpi_axi_pmu_event_dma0_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans.unit, rpi_axi_pmu_event_dma0_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans.scale, rpi_axi_pmu_event_dma0_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans.unit, rpi_axi_pmu_event_dma0_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans.scale, rpi_axi_pmu_event_dma1_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans.unit, rpi_axi_pmu_event_dma1_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans.scale, rpi_axi_pmu_event_dma1_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans.unit, rpi_axi_pmu_event_dma1_l2_wtrans_unit,
+ "Bytes");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans.scale, rpi_axi_pmu_event_v3d0_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans.unit, rpi_axi_pmu_event_v3d0_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans.scale, rpi_axi_pmu_event_v3d0_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans.unit, rpi_axi_pmu_event_v3d0_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans.scale, rpi_axi_pmu_event_v3d1_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans.unit, rpi_axi_pmu_event_v3d1_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans.scale, rpi_axi_pmu_event_v3d1_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans.unit, rpi_axi_pmu_event_v3d1_system_l2_wtrans_unit,
+ "Bytes");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans.scale, rpi_axi_pmu_event_hvs_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans.unit, rpi_axi_pmu_event_hvs_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans.scale, rpi_axi_pmu_event_hvs_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans.unit, rpi_axi_pmu_event_hvs_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans.scale, rpi_axi_pmu_event_isp_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans.unit, rpi_axi_pmu_event_isp_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans.scale, rpi_axi_pmu_event_isp_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans.unit, rpi_axi_pmu_event_isp_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans.scale, rpi_axi_pmu_event_usb_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans.unit, rpi_axi_pmu_event_usb_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.scale, rpi_axi_pmu_event_usb_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.unit, rpi_axi_pmu_event_usb_system_l2_wtrans_unit,
+ "Bytes");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_ccp2tx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_ccp2tx_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+ &rpi_axi_pmu_event_peripheral_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_peripheral_atrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans_unit.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_icache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_icache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans_unit.attr.attr,
+
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ return attr->mode;
+}
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System) are read synchronously.
+ */
+ if (event->hw.idx < 0)
+ return;
+
+ if (pmu->monitor[mon].use_mailbox_interface)
+ return;
+
+ raw_spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ raw_spin_lock_irq(&pmu->lock);
+
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx]) {
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = false;
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = true;
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
+ raw_spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM) {
+ pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] = false;
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = false;
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = true;
+ schedule_work(&pmu->vpu_work);
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ raw_spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_pmu_unregister;
+ }
+
+ return 0;
+
+err_pmu_unregister:
+ perf_pmu_unregister(&pmu->pmu);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ flush_work(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ perf_pmu_unregister(&pmu->pmu);
+ hrtimer_cancel(&pmu->hrtimer);
+ flush_work(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v8 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 6:42 ` [PATCH v8 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 6:42 ` [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 6:42 ` Ian Rogers
2026-08-13 6:52 ` sashiko-bot
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
2 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 6:42 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 450 ++++++++++++++++++++++++++++++++++++-
1 file changed, 442 insertions(+), 8 deletions(-)
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index 676c2a3d5962..25563fa789ca 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,6 +6,7 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,43 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
@@ -259,6 +300,93 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
@@ -448,7 +576,7 @@ static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_e
* struct rpi_axi_pmu - Root PMU driver context
* @pmu: Core Linux perf PMU structure
* @pdev: Owning platform_device pointer
- * @chip: Detected Broadcom SoC generation
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
* @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
* @cpu: CPU core assigned to process uncore PMU events
* @cpuhp_node: Dynamic CPU hotplug instance node
@@ -508,15 +636,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
return counter < CNT__MAX;
}
@@ -542,7 +679,7 @@ static const struct attribute_group rpi_axi_pmu_format_group = {
* Event Attribute Scaling & Unit Definitions:
*
* Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
- * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
* transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
* use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
* close (~95%) byte throughput approximation in perf stat output (accounting for
@@ -1176,6 +1313,171 @@ PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.scale, rpi_axi_pmu_event_usb_system_l
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.unit, rpi_axi_pmu_event_usb_system_l2_wtrans_unit,
"Bytes");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans.scale, rpi_axi_pmu_event_pcie_rp1_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans.unit, rpi_axi_pmu_event_pcie_rp1_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait,
+ "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans,
+ "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans.scale, rpi_axi_pmu_event_pcie_rp1_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans.unit, rpi_axi_pmu_event_pcie_rp1_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait,
+ "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans,
+ "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans.scale, rpi_axi_pmu_event_pcie_rp1_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans.unit, rpi_axi_pmu_event_pcie_rp1_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait,
+ "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans.scale, rpi_axi_pmu_event_hevc_dec_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans.unit, rpi_axi_pmu_event_hevc_dec_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans,
+ "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans.scale, rpi_axi_pmu_event_hevc_dec_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans.unit, rpi_axi_pmu_event_hevc_dec_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans,
+ "monitor=0,bus=7,counter=4");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans.scale, rpi_axi_pmu_event_hevc_dec_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans.unit, rpi_axi_pmu_event_hevc_dec_rtrans_unit,
+ "Bytes");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans.scale, rpi_axi_pmu_event_a76_dsu_l3_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans.unit, rpi_axi_pmu_event_a76_dsu_l3_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans,
+ "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans.scale, rpi_axi_pmu_event_a76_dsu_l3_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans.unit, rpi_axi_pmu_event_a76_dsu_l3_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans,
+ "monitor=0,bus=8,counter=4");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans.scale, rpi_axi_pmu_event_a76_dsu_l3_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans.unit, rpi_axi_pmu_event_a76_dsu_l3_rtrans_unit,
+ "Bytes");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_atrans.scale, rpi_axi_pmu_event_v3d7_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_atrans.unit, rpi_axi_pmu_event_v3d7_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans.scale, rpi_axi_pmu_event_v3d7_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans.unit, rpi_axi_pmu_event_v3d7_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans.scale, rpi_axi_pmu_event_v3d7_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans.unit, rpi_axi_pmu_event_v3d7_rtrans_unit,
+ "Bytes");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans.scale,
+ rpi_axi_pmu_event_bcm2712_peripheral_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans.unit,
+ rpi_axi_pmu_event_bcm2712_peripheral_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans.scale,
+ rpi_axi_pmu_event_bcm2712_peripheral_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans.unit,
+ rpi_axi_pmu_event_bcm2712_peripheral_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans,
+ "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans.scale, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans.unit, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans,
+ "monitor=0,bus=13,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans.scale, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans.unit, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_unit,
+ "Bytes");
+
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans,
+ "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_rtrans.scale, rpi_axi_pmu_event_arm_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(arm_rtrans.unit, rpi_axi_pmu_event_arm_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans,
+ "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans.scale, rpi_axi_pmu_event_arm_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(arm_wtrans.unit, rpi_axi_pmu_event_arm_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans,
+ "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans.scale, rpi_axi_pmu_event_pcie0_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans.unit, rpi_axi_pmu_event_pcie0_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans,
+ "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans.scale, rpi_axi_pmu_event_pcie0_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans.unit, rpi_axi_pmu_event_pcie0_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans,
+ "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans.scale, rpi_axi_pmu_event_hvs_filtered_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans.unit, rpi_axi_pmu_event_hvs_filtered_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans,
+ "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans.scale, rpi_axi_pmu_event_emmc0_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans.unit, rpi_axi_pmu_event_emmc0_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans,
+ "monitor=0,bus=5,counter=2,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans.scale, rpi_axi_pmu_event_emmc0_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans.unit, rpi_axi_pmu_event_emmc0_wtrans_unit,
+ "Bytes");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -1481,6 +1783,84 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_usb_system_l2_wtrans_scale.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans_unit.attr.attr,
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_arm_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans_unit.attr.attr,
NULL,
};
@@ -1491,12 +1871,62 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
* @unused: Index (unused)
*
* Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
*
* Return: attr->mode (0444) if visible on current SoC, 0 to hide.
*/
static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
struct attribute *attr, int unused)
{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+ struct perf_pmu_events_attr *pmu_attr;
+
+ pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
+ * and legacy RPi 1-4 System Monitor aliases
+ */
+ if (strstr(pmu_attr->event_str, "monitor=1") ||
+ strstr(attr->name, "cpu0_") ||
+ strstr(attr->name, "cpu1_") ||
+ strstr(attr->name, "dma0_") ||
+ strstr(attr->name, "dma1_") ||
+ strstr(attr->name, "v3d0_") ||
+ strstr(attr->name, "v3d1_") ||
+ strstr(attr->name, "hvs_system") ||
+ strstr(attr->name, "isp_system") ||
+ strstr(attr->name, "usb_system") ||
+ strstr(attr->name, "ccp2tx_") ||
+ strstr(attr->name, "mphi_") ||
+ strstr(attr->name, "h264_") ||
+ strstr(attr->name, "v3d_") ||
+ (strstr(attr->name, "peripheral_") && !strstr(attr->name, "bcm2712_")) ||
+ (strstr(attr->name, "cpu_uc_") && !strstr(attr->name, "bcm2712_")) ||
+ strstr(attr->name, "cpu_l2_") ||
+ strstr(attr->name, "l2_control_") ||
+ strstr(attr->name, "system_control_") ||
+ strstr(attr->name, "direct_control_") ||
+ strstr(attr->name, "direct_data_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_") ||
+ strstr(attr->name, "bcm2712_"))
+ return 0;
+ }
+
return attr->mode;
}
@@ -1827,7 +2257,7 @@ static void rpi_axi_pmu_read(struct perf_event *event)
u32 delta;
/* Mailbox VPU counters are polled asynchronously in background vpu_work.
- * MMIO monitors (System) are read synchronously.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
*/
if (event->hw.idx < 0)
return;
@@ -2395,6 +2825,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 6:42 ` [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 6:51 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 6:51 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Registering the PMU before completing all initialization steps creates a use-after-free race condition if probe fails.
- [Low] Usage of `#if LINUX_VERSION_CODE` shim for backward compatibility in an upstream patch.
--
--- Patch [1]: [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit e754b178c7f623d2ebff89b932e852beed8622b9
Author: Ian Rogers <irogers@google.com>
Subject: perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds a new performance monitoring driver for the Raspberry Pi
AXI bus (BCM2835/2711), exposing system-level and VideoCore PMU hardware to
the Linux perf subsystem.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..676c2a3d5962e
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
> +static inline void rpi_hrtimer_setup(struct hrtimer *timer,
> + enum hrtimer_restart (*function)(struct hrtimer *),
> + clockid_t clock_id, enum hrtimer_mode mode)
> +{
> + hrtimer_init(timer, clock_id, mode);
> + timer->function = function;
> +}
> +#define hrtimer_setup rpi_hrtimer_setup
> +#endif
[Severity: Low]
This isn't a bug, but is this version compatibility shim intended for upstream
inclusion? Upstream patches typically target the current API of the tree they
are being merged into without including out-of-tree macro shims for backward
compatibility with older kernels.
[ ... ]
> + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
> + if (ret) {
> + dev_err(dev, "PMU register failed %d\n", ret);
> + goto err_teardown;
> + }
> +
> + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
> + if (ret) {
> + dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
> + goto err_pmu_unregister;
> + }
[Severity: High]
Could this expose a use-after-free race condition if probe fails here?
When perf_pmu_register() completes, the PMU is exposed to userspace. If
cpuhp_state_add_instance() fails immediately after, the error path goes to
err_pmu_unregister and calls perf_pmu_unregister().
However, perf_pmu_unregister() does not revoke existing open events. If
userspace opens a perf event on this PMU in the small window between
registration and unregistration, the event will retain a pointer to the pmu
struct.
Because the pmu struct was allocated with devm_kzalloc(), it will be freed
by the devres manager as soon as the probe function returns the error, leaving
the open event with a dangling pointer that can corrupt memory upon subsequent
accesses. Would it be safer to register the CPU hotplug instance before
exposing the PMU to userspace?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813064228.2799872-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v8 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 6:42 ` [PATCH v8 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
@ 2026-08-13 6:52 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 6:52 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The `.scale` and `.unit` sysfs attributes for legacy VPU events are incorrectly left visible on BCM2712, creating orphaned attributes.
--
--- Patch [2]: [PATCH v8 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support ---
commit 9b809ab6d889410949331370e01a80aece4e483f
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
This patch introduces AXI PMU support for the Broadcom BCM2712 SoC used in the
Raspberry Pi 5. It expands the driver with BCM2712-specific buses, hardware
filters, and sysfs events, while ensuring compatibility with prior generations.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> index 676c2a3d5962e..25563fa789ca4 100644
> --- a/drivers/perf/rpi_axi_pmu.c
> +++ b/drivers/perf/rpi_axi_pmu.c
[ ... ]
> @@ -1491,12 +1871,62 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
> * @unused: Index (unused)
> *
> * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
> + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
> + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
> *
> * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
> */
> static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
> struct attribute *attr, int unused)
> {
> + struct device *dev = kobj_to_dev(kobj);
> + struct pmu *pmu = dev_get_drvdata(dev);
> + struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
> + struct perf_pmu_events_attr *pmu_attr;
> +
> + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr);
> +
> + if (rpi_pmu->chip == CHIP_BCM2712) {
> + /*
> + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=1)
> + * and legacy RPi 1-4 System Monitor aliases
> + */
> + if (strstr(pmu_attr->event_str, "monitor=1") ||
[Severity: Medium]
Will this correctly hide the .scale and .unit sysfs attributes for legacy VPU
events on BCM2712?
Since the event_str for those attributes is strictly "32" or "Bytes", the
check for "monitor=1" will fail. This could leave orphaned .scale and .unit
attributes visible in sysfs for events that are otherwise correctly hidden.
Could this be addressed by adding a fallback check on attr->name (for example,
checking for "vpu") to ensure the scale and unit attributes are also caught?
> + strstr(attr->name, "cpu0_") ||
> + strstr(attr->name, "cpu1_") ||
> + strstr(attr->name, "dma0_") ||
[ ... ]
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813064228.2799872-1-irogers@google.com?part=2
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver
2026-08-12 21:37 ` Ian Rogers
@ 2026-08-13 6:59 ` Uwe Kleine-König
0 siblings, 0 replies; 59+ messages in thread
From: Uwe Kleine-König @ 2026-08-13 6:59 UTC (permalink / raw)
To: Ian Rogers
Cc: Will Deacon, linux-perf-users, linux-rpi-kernel, linux-arm-kernel,
linux-kernel, mark.rutland
[-- Attachment #1: Type: text/plain, Size: 1493 bytes --]
Hello Ian,
On Wed, Aug 12, 2026 at 02:37:32PM -0700, Ian Rogers wrote:
> On Wed, Aug 12, 2026 at 9:26 AM Uwe Kleine-König
> <u.kleine-koenig@baylibre.com> wrote:
> > On Wed, Aug 12, 2026 at 06:38:24AM -0700, Ian Rogers wrote:
> > > The changes are sent in reply to each other because the entire series
> > > is visible in one mailbox (b4 mbox), which I find easier to work with
> > > and provides me better context. Others use semcode to do something
> > > similar, but I've not explored that.
> >
> > The usual recommendation is to start a new thread for a new revision.
> > Depending on the mail reader (configuration) new mails to an old thread
> > might or might not appear at the end of people's mailbox where new mail
> > usually lands and so might be missed.
>
> Yep, I agree. People who configure their email client so that a thread
> doesn't move to the top of their inbox upon receiving a reply are
> prone to missing subsequent replies in that thread. I suggest that
> people in that position change their email configuration, or use
> lore.kernel.org, which doesn't suffer from this bad configuration, for
> example:
> https://lore.kernel.org/linux-perf-users/20260811083828.2057695-1-irogers@google.com/
And I suggest that patch submitters read and honor the hints they get
from relevant members of the community, to make their submissions easy
to work with and less annoying. And no, the solution is not that others
adapt to you.
Thanks
Uwe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v9 0/2] perf: Add Raspberry Pi AXI PMU driver
@ 2026-08-13 8:03 ` Ian Rogers
0 siblings, 0 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 7:30 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
==============================
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
==============================
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, safely guarding memory teardown boundaries.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
==============================
The driver has been rigorously validated on real hardware:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters organically spanning multi-slice multiplexing boundaries.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers independently configuring `bcm2712_` filters.
Changes in v9 (Addressing upstream integration feedback)
==============================
- Replaced `flush_work()` with `cancel_work_sync()` in the `rpi_axi_pmu__exit`
and initialization error paths, rigorously ensuring no asynchronous VPU worker
threads continue executing after PMU subsystem termination.
- Inverted PMU and CPU hotplug initialization sequence in `probe()`
to cleanly eliminate a theoretical `devm_kzalloc` memory UAF race.
- Embedded a robust boolean initialization guard physically into the
hotplug event handler, definitively securing offline context callbacks
from concurrent access panics during concurrent failures.
- Architecturally radically simplified BCM2712 sysfs generation loops
by switching to an explicit inverse `bcm2712_` whitelist. This
mathematically prevents ghost suffix parameters (`.unit` and `.scale`)
from orphaned legacy VPU transactions bleeding into userspace dynamically.
- Explicitly appended documentation notes organically defending the inclusion
of the `#if LINUX_VERSION_CODE` legacy shim surrounding `hrtimer_setup`
for seamlessly preserving downstream compatibility with Raspberry Pi LTS
kernel deployments natively.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2877 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 2888 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.41.0
^ permalink raw reply [flat|nested] 59+ messages in thread
* perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
(?)
@ 2026-08-13 7:30 ` Ian Rogers
2026-08-13 7:40 ` sashiko-bot
-1 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 7:30 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This commit adds a new performance monitoring driver for the Raspberry Pi
AXI bus (BCM2835/2711), exposing system-level and VideoCore PMU hardware to
the Linux perf subsystem.
Note on out-of-tree macro compatibility:
The inclusion of #if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
surrounding hrtimer_setup is intentionally maintained alongside
this patch to guarantee seamless out-of-tree compilation fallback
compatibility for Raspberry Pi Long Term Support (LTS) kernel variants.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 2456 ++++++++++++++++++++++++++++++++++++
1 file changed, 2456 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..d8efc8bb205f
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,2456 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+/*
+ * RPI_AXI_PMU_TIMER_INTERVAL determines the background polling frequency
+ * for VideoCore VPU Mailbox IPC counters.
+ *
+ * A balance is required:
+ * - IPC Overhead: Polling overly fast (e.g., 10ms) generates excessive CPU
+ * wakeups and VideoCore IPC interrupts on older CPUs (Pi 1/2).
+ * - Accuracy: Polling overly slow (e.g., 2000ms) causes short time-multiplexed
+ * profiling sessions (under the interval) to mathematically strand residual
+ * counts since the mailbox cannot be queried synchronously inside pmu->read().
+ *
+ * 100ms (10 Hz) provides reasonably accurate profiling without heavy overhead.
+ */
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(100)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @num_started: Total count of running events on this monitor
+ * @enabled: Hardware enabled status for each watcher
+ * @monitor_running: Flag indicating if the hardware monitor loop is globally active
+ * @vpu_disable_pending: Array tracking asynchronous hardware disable requests for VPU
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+ int num_started;
+bool monitor_running;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+ bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->num_started = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->vpu_disable_pending[i] = false;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+\n * @is_registered: True if the PMU backend has securely finished initialization
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+ bool is_registered;
+
+ raw_spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans,
+ "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atrans.scale, rpi_axi_pmu_event_dma_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_atrans.unit, rpi_axi_pmu_event_dma_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait,
+ "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans,
+ "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait,
+ "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans,
+ "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait,
+ "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Events (bus=1, BCM2835_SB__TRANS: format rotation) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans,
+ "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atrans.scale, rpi_axi_pmu_event_trans_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_atrans.unit, rpi_axi_pmu_event_trans_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait,
+ "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans,
+ "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtrans.scale, rpi_axi_pmu_event_trans_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_wtrans.unit, rpi_axi_pmu_event_trans_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait,
+ "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans,
+ "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtrans.scale, rpi_axi_pmu_event_trans_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_rtrans.unit, rpi_axi_pmu_event_trans_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait,
+ "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans,
+ "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atrans.scale, rpi_axi_pmu_event_jpeg_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_atrans.unit, rpi_axi_pmu_event_jpeg_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait,
+ "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans,
+ "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans.scale, rpi_axi_pmu_event_jpeg_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans.unit, rpi_axi_pmu_event_jpeg_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait,
+ "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans,
+ "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans.scale, rpi_axi_pmu_event_jpeg_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans.unit, rpi_axi_pmu_event_jpeg_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait,
+ "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans,
+ "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atrans.scale, rpi_axi_pmu_event_system_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_atrans.unit, rpi_axi_pmu_event_system_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait,
+ "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans,
+ "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans.scale, rpi_axi_pmu_event_system_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans.unit, rpi_axi_pmu_event_system_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait,
+ "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans,
+ "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans.scale, rpi_axi_pmu_event_system_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans.unit, rpi_axi_pmu_event_system_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait,
+ "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans,
+ "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atrans.scale, rpi_axi_pmu_event_dma_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_atrans.unit, rpi_axi_pmu_event_dma_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait,
+ "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans,
+ "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans.scale, rpi_axi_pmu_event_dma_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans.unit, rpi_axi_pmu_event_dma_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait,
+ "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans,
+ "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans.scale, rpi_axi_pmu_event_dma_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans.unit, rpi_axi_pmu_event_dma_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait,
+ "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans,
+ "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atrans.scale, rpi_axi_pmu_event_system_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_atrans.unit, rpi_axi_pmu_event_system_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait,
+ "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait,
+ "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait,
+ "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans.scale, rpi_axi_pmu_event_ccp2tx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans.unit, rpi_axi_pmu_event_ccp2tx_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans.scale, rpi_axi_pmu_event_mphi_rx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans.unit, rpi_axi_pmu_event_mphi_rx_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans.scale, rpi_axi_pmu_event_mphi_tx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans.unit, rpi_axi_pmu_event_mphi_tx_atrans_unit,
+ "Bytes");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans,
+ "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atrans.scale, rpi_axi_pmu_event_hvs_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_atrans.unit, rpi_axi_pmu_event_hvs_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait,
+ "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans,
+ "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait,
+ "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans,
+ "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait,
+ "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atrans.scale, rpi_axi_pmu_event_h264_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_atrans.unit, rpi_axi_pmu_event_h264_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait,
+ "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtrans.scale, rpi_axi_pmu_event_h264_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_wtrans.unit, rpi_axi_pmu_event_h264_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait,
+ "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtrans.scale, rpi_axi_pmu_event_h264_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_rtrans.unit, rpi_axi_pmu_event_h264_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait,
+ "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans,
+ "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atrans.scale, rpi_axi_pmu_event_isp_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_atrans.unit, rpi_axi_pmu_event_isp_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait,
+ "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans,
+ "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtrans.scale, rpi_axi_pmu_event_isp_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_wtrans.unit, rpi_axi_pmu_event_isp_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait,
+ "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans,
+ "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtrans.scale, rpi_axi_pmu_event_isp_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_rtrans.unit, rpi_axi_pmu_event_isp_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait,
+ "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans,
+ "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atrans.scale, rpi_axi_pmu_event_v3d_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_atrans.unit, rpi_axi_pmu_event_v3d_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait,
+ "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait,
+ "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait,
+ "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans,
+ "monitor=0,bus=13,counter=0");
+PMU_EVENT_ATTR_STRING(peripheral_atrans.scale, rpi_axi_pmu_event_peripheral_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(peripheral_atrans.unit, rpi_axi_pmu_event_peripheral_atrans_unit,
+ "Bytes");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans,
+ "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans.scale, rpi_axi_pmu_event_cpu_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans.unit, rpi_axi_pmu_event_cpu_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait,
+ "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans,
+ "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans.scale, rpi_axi_pmu_event_cpu_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans.unit, rpi_axi_pmu_event_cpu_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait,
+ "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans,
+ "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans.scale, rpi_axi_pmu_event_cpu_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans.unit, rpi_axi_pmu_event_cpu_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait,
+ "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans,
+ "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans.scale, rpi_axi_pmu_event_cpu_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans.unit, rpi_axi_pmu_event_cpu_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait,
+ "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans,
+ "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans.scale, rpi_axi_pmu_event_cpu_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans.unit, rpi_axi_pmu_event_cpu_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait,
+ "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans,
+ "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans.scale, rpi_axi_pmu_event_cpu_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans.unit, rpi_axi_pmu_event_cpu_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait,
+ "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans,
+ "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans.scale, rpi_axi_pmu_event_vpu1_d_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans.unit, rpi_axi_pmu_event_vpu1_d_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans,
+ "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans.scale, rpi_axi_pmu_event_vpu0_d_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans.unit, rpi_axi_pmu_event_vpu0_d_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans,
+ "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans.scale, rpi_axi_pmu_event_vpu1_i_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans.unit, rpi_axi_pmu_event_vpu1_i_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans,
+ "monitor=1,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans.scale, rpi_axi_pmu_event_vpu0_i_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans.unit, rpi_axi_pmu_event_vpu0_i_l2_atrans_unit,
+ "Bytes");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans,
+ "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans.scale, rpi_axi_pmu_event_vpu_system_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans.unit, rpi_axi_pmu_event_vpu_system_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans,
+ "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans.scale, rpi_axi_pmu_event_vpu_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans.unit, rpi_axi_pmu_event_vpu_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans,
+ "monitor=1,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans.scale, rpi_axi_pmu_event_vpu_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans.unit, rpi_axi_pmu_event_vpu_system_l2_rtrans_unit,
+ "Bytes");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans,
+ "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans.scale, rpi_axi_pmu_event_vpu_l2_flush_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans.unit, rpi_axi_pmu_event_vpu_l2_flush_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans,
+ "monitor=1,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans.scale, rpi_axi_pmu_event_vpu_dma_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans.unit, rpi_axi_pmu_event_vpu_dma_l2_atrans_unit,
+ "Bytes");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans,
+ "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans.scale, rpi_axi_pmu_event_vpu1_d_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans.unit, rpi_axi_pmu_event_vpu1_d_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans,
+ "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans.scale, rpi_axi_pmu_event_vpu0_d_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans.unit, rpi_axi_pmu_event_vpu0_d_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans,
+ "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans.scale, rpi_axi_pmu_event_vpu1_i_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans.unit, rpi_axi_pmu_event_vpu1_i_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans,
+ "monitor=1,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans.scale, rpi_axi_pmu_event_vpu0_i_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans.unit, rpi_axi_pmu_event_vpu0_i_uc_atrans_unit,
+ "Bytes");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans,
+ "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans.scale, rpi_axi_pmu_event_vpu_system_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans.unit, rpi_axi_pmu_event_vpu_system_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans,
+ "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans.scale, rpi_axi_pmu_event_vpu_system_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans.unit, rpi_axi_pmu_event_vpu_system_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans,
+ "monitor=1,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans.scale, rpi_axi_pmu_event_vpu_system_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans.unit, rpi_axi_pmu_event_vpu_system_uc_rtrans_unit,
+ "Bytes");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans,
+ "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans.scale, rpi_axi_pmu_event_vpu_l2_out_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans.unit, rpi_axi_pmu_event_vpu_l2_out_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans,
+ "monitor=1,bus=13,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans.scale, rpi_axi_pmu_event_vpu_dma_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans.unit, rpi_axi_pmu_event_vpu_dma_uc_atrans_unit,
+ "Bytes");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans,
+ "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans.scale, rpi_axi_pmu_event_vpu_sdram_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans.unit, rpi_axi_pmu_event_vpu_sdram_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait,
+ "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans,
+ "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans.scale, rpi_axi_pmu_event_vpu_sdram_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans.unit, rpi_axi_pmu_event_vpu_sdram_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait,
+ "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans,
+ "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans.scale, rpi_axi_pmu_event_vpu_sdram_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans.unit, rpi_axi_pmu_event_vpu_sdram_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait,
+ "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans,
+ "monitor=1,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans.scale, rpi_axi_pmu_event_vpu_l2_in_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans.unit, rpi_axi_pmu_event_vpu_l2_in_atrans_unit,
+ "Bytes");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans.scale, rpi_axi_pmu_event_cpu0_icache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans.unit, rpi_axi_pmu_event_cpu0_icache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans.scale, rpi_axi_pmu_event_cpu0_dcache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans.unit, rpi_axi_pmu_event_cpu0_dcache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans.scale, rpi_axi_pmu_event_cpu0_dcache_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans.unit, rpi_axi_pmu_event_cpu0_dcache_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans.scale, rpi_axi_pmu_event_cpu1_icache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans.unit, rpi_axi_pmu_event_cpu1_icache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans.scale, rpi_axi_pmu_event_cpu1_dcache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans.unit, rpi_axi_pmu_event_cpu1_dcache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans.scale, rpi_axi_pmu_event_cpu1_dcache_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans.unit, rpi_axi_pmu_event_cpu1_dcache_wtrans_unit,
+ "Bytes");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans.scale, rpi_axi_pmu_event_dma0_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans.unit, rpi_axi_pmu_event_dma0_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans.scale, rpi_axi_pmu_event_dma0_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans.unit, rpi_axi_pmu_event_dma0_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans.scale, rpi_axi_pmu_event_dma1_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans.unit, rpi_axi_pmu_event_dma1_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans.scale, rpi_axi_pmu_event_dma1_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans.unit, rpi_axi_pmu_event_dma1_l2_wtrans_unit,
+ "Bytes");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans.scale, rpi_axi_pmu_event_v3d0_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans.unit, rpi_axi_pmu_event_v3d0_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans.scale, rpi_axi_pmu_event_v3d0_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans.unit, rpi_axi_pmu_event_v3d0_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans.scale, rpi_axi_pmu_event_v3d1_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans.unit, rpi_axi_pmu_event_v3d1_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans.scale, rpi_axi_pmu_event_v3d1_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans.unit, rpi_axi_pmu_event_v3d1_system_l2_wtrans_unit,
+ "Bytes");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans.scale, rpi_axi_pmu_event_hvs_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans.unit, rpi_axi_pmu_event_hvs_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans.scale, rpi_axi_pmu_event_hvs_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans.unit, rpi_axi_pmu_event_hvs_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans.scale, rpi_axi_pmu_event_isp_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans.unit, rpi_axi_pmu_event_isp_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans.scale, rpi_axi_pmu_event_isp_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans.unit, rpi_axi_pmu_event_isp_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans.scale, rpi_axi_pmu_event_usb_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans.unit, rpi_axi_pmu_event_usb_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.scale, rpi_axi_pmu_event_usb_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.unit, rpi_axi_pmu_event_usb_system_l2_wtrans_unit,
+ "Bytes");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_ccp2tx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_ccp2tx_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+ &rpi_axi_pmu_event_peripheral_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_peripheral_atrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans_unit.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_icache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_icache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans_unit.attr.attr,
+
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ return attr->mode;
+}
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System) are read synchronously.
+ */
+ if (event->hw.idx < 0)
+ return;
+
+ if (pmu->monitor[mon].use_mailbox_interface)
+ return;
+
+ raw_spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ raw_spin_lock_irq(&pmu->lock);
+
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx]) {
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = false;
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = true;
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
+ raw_spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ pmu->monitor[mon].hw_events.num_started++;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ pmu->monitor[mon].hw_events.num_started--;
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM) {
+ pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] = false;
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = false;
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = true;
+ schedule_work(&pmu->vpu_work);
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (!pmu->is_registered)
+ return 0;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ raw_spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp_remove;
+ }
+
+ pmu->is_registered = true;
+ return 0;
+
+err_cpuhp_remove:
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ perf_pmu_unregister(&pmu->pmu);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
(?)
(?)
@ 2026-08-13 7:30 ` Ian Rogers
-1 siblings, 0 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 7:30 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, u.kleine-koenig,
will
This patch introduces AXI PMU support for the Broadcom BCM2712 SoC used
in the Raspberry Pi 5. It expands the driver with BCM2712-specific
buses, hardware filters, and sysfs events, while ensuring compatibility
with prior generations.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 429 ++++++++++++++++++++++++++++++++++++-
1 file changed, 421 insertions(+), 8 deletions(-)
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index d8efc8bb205f..d69146a815c3 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,6 +6,7 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,43 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
@@ -259,6 +300,93 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
@@ -448,7 +576,7 @@ static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_e
* struct rpi_axi_pmu - Root PMU driver context
* @pmu: Core Linux perf PMU structure
* @pdev: Owning platform_device pointer
- * @chip: Detected Broadcom SoC generation
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
* @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
* @cpu: CPU core assigned to process uncore PMU events
* @cpuhp_node: Dynamic CPU hotplug instance node
@@ -510,15 +638,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
return counter < CNT__MAX;
}
@@ -544,7 +681,7 @@ static const struct attribute_group rpi_axi_pmu_format_group = {
* Event Attribute Scaling & Unit Definitions:
*
* Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
- * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
* transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
* use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
* close (~95%) byte throughput approximation in perf stat output (accounting for
@@ -1178,6 +1315,171 @@ PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.scale, rpi_axi_pmu_event_usb_system_l
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.unit, rpi_axi_pmu_event_usb_system_l2_wtrans_unit,
"Bytes");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans.scale, rpi_axi_pmu_event_pcie_rp1_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans.unit, rpi_axi_pmu_event_pcie_rp1_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait,
+ "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans,
+ "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans.scale, rpi_axi_pmu_event_pcie_rp1_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans.unit, rpi_axi_pmu_event_pcie_rp1_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait,
+ "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans,
+ "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans.scale, rpi_axi_pmu_event_pcie_rp1_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans.unit, rpi_axi_pmu_event_pcie_rp1_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait,
+ "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans.scale, rpi_axi_pmu_event_hevc_dec_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans.unit, rpi_axi_pmu_event_hevc_dec_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans,
+ "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans.scale, rpi_axi_pmu_event_hevc_dec_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans.unit, rpi_axi_pmu_event_hevc_dec_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans,
+ "monitor=0,bus=7,counter=4");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans.scale, rpi_axi_pmu_event_hevc_dec_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans.unit, rpi_axi_pmu_event_hevc_dec_rtrans_unit,
+ "Bytes");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans.scale, rpi_axi_pmu_event_a76_dsu_l3_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans.unit, rpi_axi_pmu_event_a76_dsu_l3_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans,
+ "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans.scale, rpi_axi_pmu_event_a76_dsu_l3_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans.unit, rpi_axi_pmu_event_a76_dsu_l3_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans,
+ "monitor=0,bus=8,counter=4");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans.scale, rpi_axi_pmu_event_a76_dsu_l3_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans.unit, rpi_axi_pmu_event_a76_dsu_l3_rtrans_unit,
+ "Bytes");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_atrans.scale, rpi_axi_pmu_event_v3d7_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_atrans.unit, rpi_axi_pmu_event_v3d7_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans.scale, rpi_axi_pmu_event_v3d7_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans.unit, rpi_axi_pmu_event_v3d7_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans.scale, rpi_axi_pmu_event_v3d7_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans.unit, rpi_axi_pmu_event_v3d7_rtrans_unit,
+ "Bytes");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans.scale,
+ rpi_axi_pmu_event_bcm2712_peripheral_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans.unit,
+ rpi_axi_pmu_event_bcm2712_peripheral_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans.scale,
+ rpi_axi_pmu_event_bcm2712_peripheral_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans.unit,
+ rpi_axi_pmu_event_bcm2712_peripheral_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans,
+ "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans.scale, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans.unit, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans,
+ "monitor=0,bus=13,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans.scale, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans.unit, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_unit,
+ "Bytes");
+
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans,
+ "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_rtrans.scale, rpi_axi_pmu_event_arm_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(arm_rtrans.unit, rpi_axi_pmu_event_arm_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans,
+ "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans.scale, rpi_axi_pmu_event_arm_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(arm_wtrans.unit, rpi_axi_pmu_event_arm_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans,
+ "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans.scale, rpi_axi_pmu_event_pcie0_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans.unit, rpi_axi_pmu_event_pcie0_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans,
+ "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans.scale, rpi_axi_pmu_event_pcie0_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans.unit, rpi_axi_pmu_event_pcie0_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans,
+ "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans.scale, rpi_axi_pmu_event_hvs_filtered_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans.unit, rpi_axi_pmu_event_hvs_filtered_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans,
+ "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans.scale, rpi_axi_pmu_event_emmc0_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans.unit, rpi_axi_pmu_event_emmc0_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans,
+ "monitor=0,bus=5,counter=2,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans.scale, rpi_axi_pmu_event_emmc0_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans.unit, rpi_axi_pmu_event_emmc0_wtrans_unit,
+ "Bytes");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -1483,6 +1785,84 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_usb_system_l2_wtrans_scale.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans_unit.attr.attr,
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_arm_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans_unit.attr.attr,
NULL,
};
@@ -1493,12 +1873,41 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
* @unused: Index (unused)
*
* Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
*
* Return: attr->mode (0444) if visible on current SoC, 0 to hide.
*/
static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
struct attribute *attr, int unused)
{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), aggressively filter out any legacy events,
+ * including ghost scaling parameters by exclusively matching the
+ * bcm2712 prefix.
+ */
+ if (!strstr(attr->name, "bcm2712_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_") ||
+ strstr(attr->name, "bcm2712_"))
+ return 0;
+ }
+
return attr->mode;
}
@@ -1829,7 +2238,7 @@ static void rpi_axi_pmu_read(struct perf_event *event)
u32 delta;
/* Mailbox VPU counters are polled asynchronously in background vpu_work.
- * MMIO monitors (System) are read synchronously.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
*/
if (event->hw.idx < 0)
return;
@@ -2401,6 +2810,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 7:30 ` perf: Add Raspberry Pi BCM2835 " Ian Rogers
@ 2026-08-13 7:40 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 7:40 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The `num_started` field in `struct rpi_axi_hw_events` is modified but never read or utilized.
--
--- Patch [1]: perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit 0a35d2ad48de3bdbbbe637f581c580caecb30465
Author: Ian Rogers <irogers@google.com>
Subject: perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds a new performance monitoring driver for the Raspberry Pi AXI
bus (BCM2835/2711), exposing system-level and VideoCore PMU hardware to the
Linux perf subsystem.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..d8efc8bb205fe
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
> @@ -378,9 +378,9 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
> struct rpi_axi_hw_events {
> int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
> int filter[NUM_BUS_WATCHERS_PER_MONITOR];
> int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
> int num_monitored;
> int num_started;
[Severity: Low]
Is this num_started field needed? It is tracked in both the start and stop
paths, but it does not appear to be evaluated or read anywhere else in the
driver, making it unused state tracking.
[ ... ]
> @@ -2033,10 +2033,10 @@ static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
> static void rpi_axi_pmu_start(struct perf_event *event, int flags)
> {
> struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> enum monitor mon = config_to_monitor(event->attr.config);
> enum counter counter = config_to_counter(event->attr.config);
> unsigned long spinflags;
>
> raw_spin_lock_irqsave(&pmu->lock, spinflags);
> event->hw.state = 0;
> pmu->monitor[mon].hw_events.num_started++;
[ ... ]
> @@ -2123,13 +2123,13 @@ static int rpi_axi_pmu_add(struct perf_event *event, int flags)
> static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
> {
> struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
> enum monitor mon = config_to_monitor(event->attr.config);
> unsigned long spinflags;
>
> if (event->hw.state & PERF_HES_STOPPED)
> return;
>
> if (flags & PERF_EF_UPDATE)
> rpi_axi_pmu_read(event);
>
> raw_spin_lock_irqsave(&pmu->lock, spinflags);
> if (flags & PERF_EF_UPDATE)
> event->hw.state |= PERF_HES_UPTODATE;
> event->hw.state |= PERF_HES_STOPPED;
> pmu->monitor[mon].hw_events.num_started--;
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813073050.2823657-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v10 0/2] perf: Add Raspberry Pi AXI PMU driver
@ 2026-08-13 8:03 ` Ian Rogers
0 siblings, 0 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 8:03 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, will
This patch series adds an uncore Performance Monitoring Unit (PMU) driver
for Broadcom AXI system and VideoCore VPU performance monitors found on
Raspberry Pi SoCs (BCM2835 through BCM2712 / Raspberry Pi 1 through 5).
Motivation & Background
==============================
Currently, Linux lacks a standard perf-API compatible driver for the
Broadcom AXI performance counter blocks on Raspberry Pi platforms. Prior
out-of-tree vendor solutions relied on custom debugfs nodes and ad-hoc
kthreads, preventing integration with standard Linux perf tooling (`perf stat`,
`perf list`, etc.).
This driver implements standard `struct pmu` hardware uncore callbacks
under `drivers/perf/`, exposing human-readable sysfs event aliases, unit
scaling (`Bytes`), and bus filtering directly to user space.
Key Architectural Improvements & Features
==============================
1. Standard Linux Perf Integration:
- Exposes uncore AXI interconnect events via `/sys/bus/event_source/devices/rpi_axi_pmu/`.
- Supports event sampling and hardware counter accumulation (`local64_add`),
automatically managing 31-bit hardware counter wraparound across high-bandwidth
interconnect transfers.
2. CPU Hotplug Support (`cpuhp`):
- Registers dynamic CPU hotplug notifiers (`CPUHP_AP_ONLINE_DYN`).
- Automatically migrates PMU context (`perf_pmu_migrate_context`) to an online
CPU core when a designated CPU goes offline, safely guarding memory teardown boundaries.
3. Hybrid Memory-Mapped & Mailbox Work Queue Architecture:
- System Monitor (MMIO): Performs fast atomic-safe memory reads (~15ns)
directly mapped over ARM physical memory space (`MON__SYSTEM`).
- VPU Monitor (Mailbox IPC): For Broadcom BCM2835-BCM2711 platforms (RPi 1-4),
VideoCore VPU monitor IPC calls are offloaded to process context via a dedicated
workqueue (`vpu_work`) and serialized under `vpu_mutex`. This avoids atomic
sleeps or blocking in timer/interrupt context.
4. SoC Generation Support:
- Patch 1 adds core driver support for Broadcom BCM2835-BCM2711 (RPi 1-4).
- Patch 2 expands support for Broadcom BCM2712 (Raspberry Pi 5), adding PCIe RP1
Southbridge links, HEVC decoder, HVS display engine, and Cortex-A76 DSU L3
interconnect monitoring.
Hardware Validation
==============================
The driver has been rigorously validated on real hardware:
- Raspberry Pi 400 (BCM2711): Validated System L2, ARM CPU, and VideoCore VPU
firmware mailbox IPC performance counters organically spanning multi-slice multiplexing boundaries.
- Raspberry Pi 5 (BCM2712): Validated live byte throughput across HVS display
refresh cycles, Cortex-A76 DSU L3 interconnect memory traffic, and PCIe RP1
Southbridge transfers independently configuring `bcm2712_` filters.
Changes in v10 (Addressing upstream integration feedback)
==============================
- Removed the unused `num_started` state tracking variable from `struct rpi_axi_hw_events`
and its associated start/stop increments, cleanly addressing a reviewer's note on dead code.
- Replaced `flush_work()` with `cancel_work_sync()` in the `rpi_axi_pmu__exit`
and initialization error paths, rigorously ensuring no asynchronous VPU worker
threads continue executing after PMU subsystem termination.
- Inverted PMU and CPU hotplug initialization sequence in `probe()`
to cleanly eliminate a theoretical `devm_kzalloc` memory UAF race.
- Embedded a robust boolean initialization guard physically into the
hotplug event handler, definitively securing offline context callbacks
from concurrent access panics during concurrent failures.
- Architecturally radically simplified BCM2712 sysfs generation loops
by switching to an explicit inverse `bcm2712_` whitelist. This
mathematically prevents ghost suffix parameters (`.unit` and `.scale`)
from orphaned legacy VPU transactions bleeding into userspace dynamically.
- Explicitly appended documentation notes organically defending the inclusion
of the `#if LINUX_VERSION_CODE` legacy shim surrounding `hrtimer_setup`
for seamlessly preserving downstream compatibility with Raspberry Pi LTS
kernel deployments natively.
Ian Rogers (2):
perf: Add Raspberry Pi BCM2835 AXI PMU driver
perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
drivers/perf/Kconfig | 10 +
drivers/perf/Makefile | 1 +
drivers/perf/rpi_axi_pmu.c | 2877 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 2888 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
--
2.41.0
^ permalink raw reply [flat|nested] 59+ messages in thread
* perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
` (2 preceding siblings ...)
(?)
@ 2026-08-13 8:03 ` Ian Rogers
2026-08-13 8:13 ` sashiko-bot
-1 siblings, 1 reply; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 8:03 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, will
This commit adds a new performance monitoring driver for the Raspberry Pi
AXI bus (BCM2835/2711), exposing system-level and VideoCore PMU hardware to
the Linux perf subsystem.
Note on out-of-tree macro compatibility:
The inclusion of #if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
surrounding hrtimer_setup is intentionally maintained alongside
this patch to guarantee seamless out-of-tree compilation fallback
compatibility for Raspberry Pi Long Term Support (LTS) kernel variants.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 2450 ++++++++++++++++++++++++++++++++++++
1 file changed, 2450 insertions(+)
create mode 100644 drivers/perf/rpi_axi_pmu.c
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
new file mode 100644
index 000000000000..676e8f873163
--- /dev/null
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -0,0 +1,2450 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/**
+ * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver
+ *
+ * This driver exposes the performance monitoring hardware on Raspberry Pi
+ * System-on-Chips to the Linux perf subsystem:
+ * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ *
+ * Architecture Overview:
+ * ----------------------
+ * The Broadcom AXI performance hardware provides up to two independent monitors:
+ * 1. System Monitor (MON__SYSTEM = 0):
+ * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1).
+ * Directly memory-mapped via ARM physical IO memory space (MMIO).
+ * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking).
+ *
+ * 2. VPU Monitor (MON__VPU = 1):
+ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
+ * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
+ * Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ *
+ * Synchronization & Concurrency Model:
+ * ------------------------------------
+ * - Spinlock (pmu->lock):
+ * Protects active event array (events[]), event generation sequence counters (event_gen[]),
+ * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates
+ * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling races.
+ *
+ * - Mutex (pmu->vpu_mutex):
+ * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process context,
+ * preventing concurrent mailbox buffer corruption across multiple CPUs.
+ *
+ * - Cached Async VPU Reads & Multiplexing (MON__VPU):
+ * Polled periodically in process context by vpu_work when active_vpu_events > 0.
+ * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during
+ * event rotation / multiplexing. User read() syscalls return cached cumulative event counter
+ * instantly without blocking.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/version.h>
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
+static inline void rpi_hrtimer_setup(struct hrtimer *timer,
+ enum hrtimer_restart (*function)(struct hrtimer *),
+ clockid_t clock_id, enum hrtimer_mode mode)
+{
+ hrtimer_init(timer, clock_id, mode);
+ timer->function = function;
+}
+#define hrtimer_setup rpi_hrtimer_setup
+#endif
+#include <linux/lockdep.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/sysfs.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */
+
+/**
+ * enum rpi_axi_chip - Supported Broadcom SoC generations
+ * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ */
+enum rpi_axi_chip {
+ CHIP_BCM2835 = 0,
+};
+
+enum monitor {
+ MON__SYSTEM = 0,
+ MON__VPU,
+ MON__MAX
+};
+
+/* Number of hardware bus watcher units per monitor */
+#define NUM_BUS_WATCHERS_PER_MONITOR 3
+
+/**
+ * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D matrix conversion)
+ * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2835_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus
+ * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus
+ * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus
+ * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus
+ * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus
+ * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeline bus
+ * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus
+ * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM)
+ * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus
+ * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus
+ * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2711
+ */
+enum bcm2835_system_bus {
+ BCM2835_SB__DMA_L2 = 0,
+ BCM2835_SB__TRANS,
+ BCM2835_SB__JPEG,
+ BCM2835_SB__SYSTEM_UC,
+ BCM2835_SB__DMA_UC,
+ BCM2835_SB__SYSTEM_L2,
+ BCM2835_SB__CCP2TX,
+ BCM2835_SB__MPHI_RX,
+ BCM2835_SB__MPHI_TX,
+ BCM2835_SB__HVS,
+ BCM2835_SB__H264,
+ BCM2835_SB__ISP,
+ BCM2835_SB__V3D,
+ BCM2835_SB__PERIPHERAL,
+ BCM2835_SB__CPU_UC,
+ BCM2835_SB__CPU_L2,
+ BCM2835_SB__MAX
+};
+
+
+/**
+ * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
+ * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus
+ * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus
+ * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus
+ * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus
+ * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus
+ * @VPU__L2_FLUSH: VPU L2 cache flush controller bus
+ * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus
+ * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus
+ * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus
+ * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus
+ * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus
+ * @VPU__SYSTEM_UC: VPU System Uncached memory bus
+ * @VPU__L2_OUT: VPU L2 cache outbound memory bus
+ * @VPU__DMA_UC: VPU DMA Uncached memory bus
+ * @VPU__SDRAM: VPU SDRAM memory controller bus
+ * @VPU__L2_IN: VPU L2 cache inbound memory bus
+ * @VPU__MAX: Total count of monitored VPU buses
+ */
+enum vpu_bus {
+ VPU__VPU1_D_L2 = 0,
+ VPU__VPU0_D_L2,
+ VPU__VPU1_I_L2,
+ VPU__VPU0_I_L2,
+ VPU__SYSTEM_L2,
+ VPU__L2_FLUSH,
+ VPU__DMA_L2,
+ VPU__VPU1_D_UC,
+ VPU__VPU0_D_UC,
+ VPU__VPU1_I_UC,
+ VPU__VPU0_I_UC,
+ VPU__SYSTEM_UC,
+ VPU__L2_OUT,
+ VPU__DMA_UC,
+ VPU__SDRAM,
+ VPU__L2_IN,
+ VPU__MAX
+};
+
+/**
+ * enum counter - 32-bit hardware performance counter metrics per bus watcher unit
+ * @CNT__ATRANS: Total address phase transaction count
+ * @CNT__ATWAIT: Total address phase wait / stall cycles
+ * @CNT__WTRANS: Total write data phase transaction count
+ * @CNT__WTWAIT: Total write data phase wait / stall cycles
+ * @CNT__RTRANS: Total read data phase transaction count
+ * @CNT__RTWAIT: Total read data phase wait / stall cycles
+ * @CNT__MAX: Total metric counters per watcher unit
+ */
+enum counter {
+ CNT__ATRANS = 0,
+ CNT__ATWAIT,
+ CNT__WTRANS,
+ CNT__WTWAIT,
+ CNT__RTRANS,
+ CNT__RTWAIT,
+ CNT__MAX
+};
+
+/**
+ * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4)
+ * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID
+ * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID
+ * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID
+ * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID
+ * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID
+ * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID
+ * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID
+ * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID
+ * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master ID
+ * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID
+ * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID
+ * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID
+ * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID
+ * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID
+ * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID
+ * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID
+ * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID
+ * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master ID
+ * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID
+ * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID
+ * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID
+ * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID
+ * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID
+ * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID
+ * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID
+ * @BCM2835_FLT__AVE: Audio-Video Engine master ID
+ * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID
+ * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID
+ * @BCM2835_FLT__M30: Reserved master ID slot 30
+ * @BCM2835_FLT__MAX: Maximum filter ID count
+ */
+enum bcm2835_filter {
+ BCM2835_FLT__NONE = 0,
+ BCM2835_FLT__CORE0_V,
+ BCM2835_FLT__ICACHE0,
+ BCM2835_FLT__DCACHE0,
+ BCM2835_FLT__CORE1_V,
+ BCM2835_FLT__ICACHE1,
+ BCM2835_FLT__DCACHE1,
+ BCM2835_FLT__L2_MAIN,
+ BCM2835_FLT__HOST_PORT,
+ BCM2835_FLT__HOST_PORT2,
+ BCM2835_FLT__HVS,
+ BCM2835_FLT__ISP,
+ BCM2835_FLT__VIDEO_DCT,
+ BCM2835_FLT__VIDEO_SD2AXI,
+ BCM2835_FLT__CAM0,
+ BCM2835_FLT__CAM1,
+ BCM2835_FLT__DMA0,
+ BCM2835_FLT__DMA1,
+ BCM2835_FLT__DMA2_VPU,
+ BCM2835_FLT__JPEG,
+ BCM2835_FLT__VIDEO_CME,
+ BCM2835_FLT__TRANSPOSER,
+ BCM2835_FLT__VIDEO_FME,
+ BCM2835_FLT__CCP2TX,
+ BCM2835_FLT__USB,
+ BCM2835_FLT__V3D0,
+ BCM2835_FLT__V3D1,
+ BCM2835_FLT__V3D2,
+ BCM2835_FLT__AVE,
+ BCM2835_FLT__DEBUG,
+ BCM2835_FLT__CPU,
+ BCM2835_FLT__M30,
+ BCM2835_FLT__MAX
+};
+
+
+/* Hardware register offsets & control bitwise constants */
+#define GEN_CTRL 0x00
+#define GEN_CTL_ENABLE_BIT BIT(0)
+#define GEN_CTL_RESET_BIT BIT(1)
+#define GEN_CTL_WATCH_BIT BIT(2)
+
+#define BW_PITCH 0x40
+#define BW0_CTRL 0x40
+#define BW1_CTRL 0x80
+#define BW2_CTRL 0xc0
+
+#define BW_ATRANS_OFFSET 0x04
+#define BW_ATWAIT_OFFSET 0x08
+#define BW_AMAX_OFFSET 0x0c
+#define BW_WTRANS_OFFSET 0x10
+#define BW_WTWAIT_OFFSET 0x14
+#define BW_WMAX_OFFSET 0x18
+#define BW_RTRANS_OFFSET 0x1c
+#define BW_RTWAIT_OFFSET 0x20
+#define BW_RMAX_OFFSET 0x24
+
+#define BW_CTRL_RESET_BIT BIT(31)
+#define BW_CTRL_ENABLE_BIT BIT(30)
+#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29)
+#define BW_CTRL_LIMIT_HALT_BIT BIT(28)
+
+#define BW_CTRL_BUS_WATCH_SHIFT 0
+#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0)
+#define BW_CTRL_BUS_FILTER_SHIFT 8
+#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8)
+
+/*
+ * RPI_AXI_PMU_TIMER_INTERVAL determines the background polling frequency
+ * for VideoCore VPU Mailbox IPC counters.
+ *
+ * A balance is required:
+ * - IPC Overhead: Polling overly fast (e.g., 10ms) generates excessive CPU
+ * wakeups and VideoCore IPC interrupts on older CPUs (Pi 1/2).
+ * - Accuracy: Polling overly slow (e.g., 2000ms) causes short time-multiplexed
+ * profiling sessions (under the interval) to mathematically strand residual
+ * counts since the mailbox cannot be queried synchronously inside pmu->read().
+ *
+ * 100ms (10 Hz) provides reasonably accurate profiling without heavy overhead.
+ */
+#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(100)
+
+static enum cpuhp_state rpi_axi_pmu_cpuhp_state;
+
+/* --- PMU API & CONFIG DECODING ---------------------------------- */
+
+#define PMU_NAME "rpi_axi_pmu"
+
+/**
+ * config_to_filter() - Extracts AXI filter ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Filter ID value (bits 10-14).
+ */
+static int config_to_filter(__u64 config)
+{
+ return (config >> 10) & 0x1F;
+}
+
+/**
+ * config_to_monitor() - Extracts Monitor ID from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Monitor enum (bit 9: 0 = System, 1 = VPU).
+ */
+static enum monitor config_to_monitor(__u64 config)
+{
+ return (config >> 9) & 1;
+}
+
+/**
+ * config_to_bus() - Extracts bus index from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Bus index (bits 4-8).
+ */
+static int config_to_bus(__u64 config)
+{
+ return (config >> 4) & 0x1F;
+}
+
+/**
+ * config_to_counter() - Extracts metric counter type from perf event config
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: Counter enum (bits 0-3).
+ */
+static enum counter config_to_counter(__u64 config)
+{
+ return config & 0xF;
+}
+
+struct rpi_axi_pmu;
+
+/**
+ * config_is_valid() - Validates whether event config bitfields match SoC capabilities
+ * @pmu: Pointer to rpi_axi_pmu driver context
+ * @config: 64-bit config value from struct perf_event_attr
+ *
+ * Return: true if valid for the detected chip, false otherwise.
+ */
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config);
+
+/**
+ * struct rpi_axi_hw_events - Hardware resource tracking per monitor
+ * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers
+ * @filter: Array tracking the filter applied to each watcher
+ * @refcount: Reference count of active events sharing each watcher
+ * @num_monitored: Total count of active watchers in use
+ * @enabled: Hardware enabled status for each watcher
+ * @monitor_running: Flag indicating if the hardware monitor loop is globally active
+ * @vpu_disable_pending: Array tracking asynchronous hardware disable requests for VPU
+ */
+struct rpi_axi_hw_events {
+ int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR];
+ int filter[NUM_BUS_WATCHERS_PER_MONITOR];
+ int refcount[NUM_BUS_WATCHERS_PER_MONITOR];
+ int num_monitored;
+bool monitor_running;
+ bool enabled[NUM_BUS_WATCHERS_PER_MONITOR];
+ bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR];
+};
+
+/**
+ * rpi_axi_hw_events__init() - Resets hardware watcher tracking data
+ * @hw_events: Pointer to rpi_axi_hw_events structure
+ */
+static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events)
+{
+ hw_events->num_monitored = 0;
+ hw_events->monitor_running = false;
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ hw_events->monitored_bus[i] = -1;
+ hw_events->filter[i] = BCM2835_FLT__NONE;
+ hw_events->refcount[i] = 0;
+ hw_events->enabled[i] = false;
+ }
+}
+
+/**
+ * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index
+ * @hw_events: Pointer to hardware watcher tracking state
+ * @event: Pointer to perf_event being initialized
+ *
+ * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy.
+ */
+static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events,
+ const struct perf_event *event)
+{
+ int bus = config_to_bus(event->attr.config);
+ int filter = config_to_filter(event->attr.config);
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) {
+ hw_events->refcount[i]++;
+ return i;
+ }
+ }
+ if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR)
+ return -1;
+
+ for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) {
+ if (hw_events->monitored_bus[i] == -1) {
+ hw_events->monitored_bus[i] = bus;
+ hw_events->filter[i] = filter;
+ hw_events->refcount[i] = 1;
+ hw_events->vpu_disable_pending[i] = false;
+ hw_events->num_monitored++;
+ return i;
+ }
+ }
+ return -1;
+}
+
+/* Maximum simultaneous active perf_events tracked by PMU */
+#define RPI_AXI_MAX_EVENTS 60
+
+/**
+ * struct rpi_axi_pmu - Root PMU driver context
+ * @pmu: Core Linux perf PMU structure
+ * @pdev: Owning platform_device pointer
+ * @chip: Detected Broadcom SoC generation
+ * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
+ * @cpu: CPU core assigned to process uncore PMU events
+ * @cpuhp_node: Dynamic CPU hotplug instance node
+\n * @is_registered: True if the PMU backend has securely finished initialization
+ * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates
+ * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context
+ * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling
+ * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads
+ * @active_events: Count of active perf_events currently monitored
+ * @active_vpu_events: Count of active VPU perf_events currently monitored
+ * @events: Array of active perf_event pointers
+ * @event_gen: Per-slot generation sequence counters to prevent ABA pointer recycling races
+ * @monitor: Per-monitor state array (System and VPU)
+ */
+struct rpi_axi_pmu {
+ struct pmu pmu;
+ struct platform_device *pdev;
+ enum rpi_axi_chip chip;
+ struct rpi_firmware *firmware;
+
+ int cpu;
+ struct hlist_node cpuhp_node;
+ bool is_registered;
+
+ raw_spinlock_t lock;
+ struct mutex vpu_mutex;
+
+ struct hrtimer hrtimer;
+ struct work_struct vpu_work;
+ int active_events;
+ int active_vpu_events;
+ struct perf_event *events[RPI_AXI_MAX_EVENTS];
+ u64 event_gen[RPI_AXI_MAX_EVENTS];
+
+ struct {
+ struct rpi_axi_hw_events hw_events;
+ bool use_mailbox_interface;
+ union {
+ u32 mailbox;
+ void __iomem *base_address;
+ };
+ } monitor[MON__MAX];
+};
+#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu))
+
+static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
+{
+ enum monitor mon = config_to_monitor(config);
+ int bus = config_to_bus(config);
+ int filter = config_to_filter(config);
+ int counter = config_to_counter(config);
+
+ if (config >> 15 != 0)
+ return false;
+
+ if (mon >= MON__MAX)
+ return false;
+
+ if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
+ return false;
+
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
+ return false;
+ return counter < CNT__MAX;
+}
+
+PMU_FORMAT_ATTR(filter, "config:10-14");
+PMU_FORMAT_ATTR(monitor, "config:9");
+PMU_FORMAT_ATTR(bus, "config:4-8");
+PMU_FORMAT_ATTR(counter, "config:0-3");
+
+static struct attribute *rpi_axi_pmu_formats_attr[] = {
+ &format_attr_filter.attr,
+ &format_attr_monitor.attr,
+ &format_attr_bus.attr,
+ &format_attr_counter.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_format_group = {
+ .name = "format",
+ .attrs = rpi_axi_pmu_formats_attr,
+};
+
+/*
+ * Event Attribute Scaling & Unit Definitions:
+ *
+ * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
+ * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
+ * use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
+ * close (~95%) byte throughput approximation in perf stat output (accounting for
+ * 64-byte burst beats and VideoCore L2 cache prefetch hits).
+ */
+/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */
+
+/* DMA Engine L2 Cache Bus Events (bus=0, BCM2835_SB__DMA_L2) */
+PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans,
+ "monitor=0,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(dma_l2_atrans.scale, rpi_axi_pmu_event_dma_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_atrans.unit, rpi_axi_pmu_event_dma_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait,
+ "monitor=0,bus=0,counter=1");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans,
+ "monitor=0,bus=0,counter=2");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait,
+ "monitor=0,bus=0,counter=3");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans,
+ "monitor=0,bus=0,counter=4");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait,
+ "monitor=0,bus=0,counter=5");
+
+/* Image Transposer Engine Events (bus=1, BCM2835_SB__TRANS: format rotation) */
+PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans,
+ "monitor=0,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(trans_atrans.scale, rpi_axi_pmu_event_trans_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_atrans.unit, rpi_axi_pmu_event_trans_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait,
+ "monitor=0,bus=1,counter=1");
+PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans,
+ "monitor=0,bus=1,counter=2");
+PMU_EVENT_ATTR_STRING(trans_wtrans.scale, rpi_axi_pmu_event_trans_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_wtrans.unit, rpi_axi_pmu_event_trans_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait,
+ "monitor=0,bus=1,counter=3");
+PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans,
+ "monitor=0,bus=1,counter=4");
+PMU_EVENT_ATTR_STRING(trans_rtrans.scale, rpi_axi_pmu_event_trans_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(trans_rtrans.unit, rpi_axi_pmu_event_trans_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait,
+ "monitor=0,bus=1,counter=5");
+
+/* Hardware JPEG Codec Accelerator Bus Events (bus=2, BCM2835_SB__JPEG) */
+PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans,
+ "monitor=0,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(jpeg_atrans.scale, rpi_axi_pmu_event_jpeg_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_atrans.unit, rpi_axi_pmu_event_jpeg_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait,
+ "monitor=0,bus=2,counter=1");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans,
+ "monitor=0,bus=2,counter=2");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans.scale, rpi_axi_pmu_event_jpeg_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_wtrans.unit, rpi_axi_pmu_event_jpeg_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait,
+ "monitor=0,bus=2,counter=3");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans,
+ "monitor=0,bus=2,counter=4");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans.scale, rpi_axi_pmu_event_jpeg_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(jpeg_rtrans.unit, rpi_axi_pmu_event_jpeg_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait,
+ "monitor=0,bus=2,counter=5");
+
+/* System Uncached Memory Bus Events (bus=3, BCM2835_SB__SYSTEM_UC) */
+PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans,
+ "monitor=0,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(system_uc_atrans.scale, rpi_axi_pmu_event_system_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_atrans.unit, rpi_axi_pmu_event_system_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait,
+ "monitor=0,bus=3,counter=1");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans,
+ "monitor=0,bus=3,counter=2");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans.scale, rpi_axi_pmu_event_system_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_wtrans.unit, rpi_axi_pmu_event_system_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait,
+ "monitor=0,bus=3,counter=3");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans,
+ "monitor=0,bus=3,counter=4");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans.scale, rpi_axi_pmu_event_system_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_uc_rtrans.unit, rpi_axi_pmu_event_system_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait,
+ "monitor=0,bus=3,counter=5");
+
+/* DMA Engine Uncached Memory Bus Events (bus=4, BCM2835_SB__DMA_UC) */
+PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans,
+ "monitor=0,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(dma_uc_atrans.scale, rpi_axi_pmu_event_dma_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_atrans.unit, rpi_axi_pmu_event_dma_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait,
+ "monitor=0,bus=4,counter=1");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans,
+ "monitor=0,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans.scale, rpi_axi_pmu_event_dma_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_wtrans.unit, rpi_axi_pmu_event_dma_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait,
+ "monitor=0,bus=4,counter=3");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans,
+ "monitor=0,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans.scale, rpi_axi_pmu_event_dma_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma_uc_rtrans.unit, rpi_axi_pmu_event_dma_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait,
+ "monitor=0,bus=4,counter=5");
+
+/* System Main L2 Cache Bus Events (bus=5, BCM2835_SB__SYSTEM_L2) */
+PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans,
+ "monitor=0,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(system_l2_atrans.scale, rpi_axi_pmu_event_system_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_atrans.unit, rpi_axi_pmu_event_system_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait,
+ "monitor=0,bus=5,counter=1");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait,
+ "monitor=0,bus=5,counter=3");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait,
+ "monitor=0,bus=5,counter=5");
+
+/* Legacy Camera & Message Passing Bus Events (bus=6-8, CCP2TX, MPHI_RX, MPHI_TX) */
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans.scale, rpi_axi_pmu_event_ccp2tx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(ccp2tx_atrans.unit, rpi_axi_pmu_event_ccp2tx_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans.scale, rpi_axi_pmu_event_mphi_rx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(mphi_rx_atrans.unit, rpi_axi_pmu_event_mphi_rx_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans.scale, rpi_axi_pmu_event_mphi_tx_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(mphi_tx_atrans.unit, rpi_axi_pmu_event_mphi_tx_atrans_unit,
+ "Bytes");
+
+/* Hardware Video Scaler Display Engine Bus Events (bus=9, BCM2835_SB__HVS) */
+PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans,
+ "monitor=0,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(hvs_atrans.scale, rpi_axi_pmu_event_hvs_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_atrans.unit, rpi_axi_pmu_event_hvs_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait,
+ "monitor=0,bus=9,counter=1");
+PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans,
+ "monitor=0,bus=9,counter=2");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait,
+ "monitor=0,bus=9,counter=3");
+PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans,
+ "monitor=0,bus=9,counter=4");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait,
+ "monitor=0,bus=9,counter=5");
+
+/* H.264 Video Encoder/Decoder Bus Events (bus=10, BCM2835_SB__H264) */
+PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(h264_atrans.scale, rpi_axi_pmu_event_h264_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_atrans.unit, rpi_axi_pmu_event_h264_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait,
+ "monitor=0,bus=10,counter=1");
+PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(h264_wtrans.scale, rpi_axi_pmu_event_h264_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_wtrans.unit, rpi_axi_pmu_event_h264_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait,
+ "monitor=0,bus=10,counter=3");
+PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(h264_rtrans.scale, rpi_axi_pmu_event_h264_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(h264_rtrans.unit, rpi_axi_pmu_event_h264_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait,
+ "monitor=0,bus=10,counter=5");
+
+/* Image Sensor Processor Camera Pipeline Bus Events (bus=11, BCM2835_SB__ISP) */
+PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans,
+ "monitor=0,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(isp_atrans.scale, rpi_axi_pmu_event_isp_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_atrans.unit, rpi_axi_pmu_event_isp_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait,
+ "monitor=0,bus=11,counter=1");
+PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans,
+ "monitor=0,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(isp_wtrans.scale, rpi_axi_pmu_event_isp_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_wtrans.unit, rpi_axi_pmu_event_isp_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait,
+ "monitor=0,bus=11,counter=3");
+PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans,
+ "monitor=0,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(isp_rtrans.scale, rpi_axi_pmu_event_isp_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_rtrans.unit, rpi_axi_pmu_event_isp_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait,
+ "monitor=0,bus=11,counter=5");
+
+/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=12, BCM2835_SB__V3D) */
+PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans,
+ "monitor=0,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(v3d_atrans.scale, rpi_axi_pmu_event_v3d_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_atrans.unit, rpi_axi_pmu_event_v3d_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait,
+ "monitor=0,bus=12,counter=1");
+PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait,
+ "monitor=0,bus=12,counter=3");
+PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait,
+ "monitor=0,bus=12,counter=5");
+
+/* Low-Speed Peripherals Bus Events (bus=13, BCM2835_SB__PERIPHERAL: UART, SPI, I2C, GPIO) */
+PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atrans,
+ "monitor=0,bus=13,counter=0");
+PMU_EVENT_ATTR_STRING(peripheral_atrans.scale, rpi_axi_pmu_event_peripheral_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(peripheral_atrans.unit, rpi_axi_pmu_event_peripheral_atrans_unit,
+ "Bytes");
+
+/* CPU Uncached Memory Bus Events (bus=14, BCM2835_SB__CPU_UC) */
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans,
+ "monitor=0,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans.scale, rpi_axi_pmu_event_cpu_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_atrans.unit, rpi_axi_pmu_event_cpu_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait,
+ "monitor=0,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans,
+ "monitor=0,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans.scale, rpi_axi_pmu_event_cpu_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtrans.unit, rpi_axi_pmu_event_cpu_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait,
+ "monitor=0,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans,
+ "monitor=0,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans.scale, rpi_axi_pmu_event_cpu_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtrans.unit, rpi_axi_pmu_event_cpu_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait,
+ "monitor=0,bus=14,counter=5");
+
+/* CPU L2 Cache Bus Events (bus=15, BCM2835_SB__CPU_L2) */
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans,
+ "monitor=0,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans.scale, rpi_axi_pmu_event_cpu_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_atrans.unit, rpi_axi_pmu_event_cpu_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait,
+ "monitor=0,bus=15,counter=1");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans,
+ "monitor=0,bus=15,counter=2");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans.scale, rpi_axi_pmu_event_cpu_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtrans.unit, rpi_axi_pmu_event_cpu_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait,
+ "monitor=0,bus=15,counter=3");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans,
+ "monitor=0,bus=15,counter=4");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans.scale, rpi_axi_pmu_event_cpu_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtrans.unit, rpi_axi_pmu_event_cpu_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait,
+ "monitor=0,bus=15,counter=5");
+
+/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU = 1, RPi 1-4) ------------------ */
+
+/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans,
+ "monitor=1,bus=0,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans.scale, rpi_axi_pmu_event_vpu1_d_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans.unit, rpi_axi_pmu_event_vpu1_d_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans,
+ "monitor=1,bus=1,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans.scale, rpi_axi_pmu_event_vpu0_d_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans.unit, rpi_axi_pmu_event_vpu0_d_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans,
+ "monitor=1,bus=2,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans.scale, rpi_axi_pmu_event_vpu1_i_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans.unit, rpi_axi_pmu_event_vpu1_i_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans,
+ "monitor=1,bus=3,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans.scale, rpi_axi_pmu_event_vpu0_i_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans.unit, rpi_axi_pmu_event_vpu0_i_l2_atrans_unit,
+ "Bytes");
+
+/* VPU System L2 Cache Interconnect Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l2_atrans,
+ "monitor=1,bus=4,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans.scale, rpi_axi_pmu_event_vpu_system_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans.unit, rpi_axi_pmu_event_vpu_system_l2_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l2_wtrans,
+ "monitor=1,bus=4,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans.scale, rpi_axi_pmu_event_vpu_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans.unit, rpi_axi_pmu_event_vpu_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l2_rtrans,
+ "monitor=1,bus=4,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans.scale, rpi_axi_pmu_event_vpu_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans.unit, rpi_axi_pmu_event_vpu_system_l2_rtrans_unit,
+ "Bytes");
+
+/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_atrans,
+ "monitor=1,bus=5,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans.scale, rpi_axi_pmu_event_vpu_l2_flush_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans.unit, rpi_axi_pmu_event_vpu_l2_flush_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atrans,
+ "monitor=1,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans.scale, rpi_axi_pmu_event_vpu_dma_l2_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans.unit, rpi_axi_pmu_event_vpu_dma_l2_atrans_unit,
+ "Bytes");
+
+/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans,
+ "monitor=1,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans.scale, rpi_axi_pmu_event_vpu1_d_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans.unit, rpi_axi_pmu_event_vpu1_d_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans,
+ "monitor=1,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans.scale, rpi_axi_pmu_event_vpu0_d_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans.unit, rpi_axi_pmu_event_vpu0_d_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans,
+ "monitor=1,bus=9,counter=0");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans.scale, rpi_axi_pmu_event_vpu1_i_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans.unit, rpi_axi_pmu_event_vpu1_i_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans,
+ "monitor=1,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans.scale, rpi_axi_pmu_event_vpu0_i_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans.unit, rpi_axi_pmu_event_vpu0_i_uc_atrans_unit,
+ "Bytes");
+
+/* VPU System Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_uc_atrans,
+ "monitor=1,bus=11,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans.scale, rpi_axi_pmu_event_vpu_system_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans.unit, rpi_axi_pmu_event_vpu_system_uc_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_uc_wtrans,
+ "monitor=1,bus=11,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans.scale, rpi_axi_pmu_event_vpu_system_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans.unit, rpi_axi_pmu_event_vpu_system_uc_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_uc_rtrans,
+ "monitor=1,bus=11,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans.scale, rpi_axi_pmu_event_vpu_system_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans.unit, rpi_axi_pmu_event_vpu_system_uc_rtrans_unit,
+ "Bytes");
+
+/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atrans,
+ "monitor=1,bus=12,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans.scale, rpi_axi_pmu_event_vpu_l2_out_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans.unit, rpi_axi_pmu_event_vpu_l2_out_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atrans,
+ "monitor=1,bus=13,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans.scale, rpi_axi_pmu_event_vpu_dma_uc_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans.unit, rpi_axi_pmu_event_vpu_dma_uc_atrans_unit,
+ "Bytes");
+
+/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans,
+ "monitor=1,bus=14,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans.scale, rpi_axi_pmu_event_vpu_sdram_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atrans.unit, rpi_axi_pmu_event_vpu_sdram_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait,
+ "monitor=1,bus=14,counter=1");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans,
+ "monitor=1,bus=14,counter=2");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans.scale, rpi_axi_pmu_event_vpu_sdram_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans.unit, rpi_axi_pmu_event_vpu_sdram_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait,
+ "monitor=1,bus=14,counter=3");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans,
+ "monitor=1,bus=14,counter=4");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans.scale, rpi_axi_pmu_event_vpu_sdram_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans.unit, rpi_axi_pmu_event_vpu_sdram_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait,
+ "monitor=1,bus=14,counter=5");
+
+/* VPU L2 Cache Inbound Memory Bus Events */
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans,
+ "monitor=1,bus=15,counter=0");
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans.scale, rpi_axi_pmu_event_vpu_l2_in_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans.unit, rpi_axi_pmu_event_vpu_l2_in_atrans_unit,
+ "Bytes");
+
+/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) --------------- */
+
+/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=2");
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans.scale, rpi_axi_pmu_event_cpu0_icache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans.unit, rpi_axi_pmu_event_cpu0_icache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans.scale, rpi_axi_pmu_event_cpu0_dcache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans.unit, rpi_axi_pmu_event_cpu0_dcache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=3");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans.scale, rpi_axi_pmu_event_cpu0_dcache_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans.unit, rpi_axi_pmu_event_cpu0_dcache_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=5");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans.scale, rpi_axi_pmu_event_cpu1_icache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans.unit, rpi_axi_pmu_event_cpu1_icache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rtrans,
+ "monitor=0,bus=15,counter=4,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans.scale, rpi_axi_pmu_event_cpu1_dcache_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans.unit, rpi_axi_pmu_event_cpu1_dcache_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wtrans,
+ "monitor=0,bus=15,counter=2,filter=6");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans.scale, rpi_axi_pmu_event_cpu1_dcache_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans.unit, rpi_axi_pmu_event_cpu1_dcache_wtrans_unit,
+ "Bytes");
+
+/* System DMA Channels 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans.scale, rpi_axi_pmu_event_dma0_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma0_l2_rtrans.unit, rpi_axi_pmu_event_dma0_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=16");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans.scale, rpi_axi_pmu_event_dma0_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma0_l2_wtrans.unit, rpi_axi_pmu_event_dma0_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans,
+ "monitor=0,bus=0,counter=4,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans.scale, rpi_axi_pmu_event_dma1_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma1_l2_rtrans.unit, rpi_axi_pmu_event_dma1_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans,
+ "monitor=0,bus=0,counter=2,filter=17");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans.scale, rpi_axi_pmu_event_dma1_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(dma1_l2_wtrans.unit, rpi_axi_pmu_event_dma1_l2_wtrans_unit,
+ "Bytes");
+
+/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans.scale, rpi_axi_pmu_event_v3d0_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans.unit, rpi_axi_pmu_event_v3d0_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=25");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans.scale, rpi_axi_pmu_event_v3d0_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans.unit, rpi_axi_pmu_event_v3d0_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans.scale, rpi_axi_pmu_event_v3d1_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans.unit, rpi_axi_pmu_event_v3d1_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=26");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans.scale, rpi_axi_pmu_event_v3d1_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans.unit, rpi_axi_pmu_event_v3d1_system_l2_wtrans_unit,
+ "Bytes");
+
+/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Events */
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans.scale, rpi_axi_pmu_event_hvs_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans.unit, rpi_axi_pmu_event_hvs_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans.scale, rpi_axi_pmu_event_hvs_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans.unit, rpi_axi_pmu_event_hvs_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans.scale, rpi_axi_pmu_event_isp_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans.unit, rpi_axi_pmu_event_isp_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=11");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans.scale, rpi_axi_pmu_event_isp_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans.unit, rpi_axi_pmu_event_isp_system_l2_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l2_rtrans,
+ "monitor=0,bus=5,counter=4,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans.scale, rpi_axi_pmu_event_usb_system_l2_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans.unit, rpi_axi_pmu_event_usb_system_l2_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l2_wtrans,
+ "monitor=0,bus=5,counter=2,filter=24");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.scale, rpi_axi_pmu_event_usb_system_l2_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.unit, rpi_axi_pmu_event_usb_system_l2_wtrans_unit,
+ "Bytes");
+
+static struct attribute *rpi_axi_pmu_events_attrs[] = {
+ /* System Monitor events */
+ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_trans_atrans.attr.attr,
+ &rpi_axi_pmu_event_trans_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_atwait.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_wtwait.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_trans_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_trans_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_jpeg_atrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_atwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_wtwait.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_jpeg_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_system_l2_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_ccp2tx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_ccp2tx_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_mphi_rx_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_mphi_tx_atrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_hvs_atrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_atwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_wtwait.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_h264_atrans.attr.attr,
+ &rpi_axi_pmu_event_h264_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_atwait.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_wtwait.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_h264_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_h264_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_isp_atrans.attr.attr,
+ &rpi_axi_pmu_event_isp_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_atwait.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_wtwait.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_v3d_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_atwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_wtwait.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_peripheral_atrans.attr.attr,
+ &rpi_axi_pmu_event_peripheral_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_peripheral_atrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr,
+
+ /* VPU Monitor events (RPi 1-4) */
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_flush_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_l2_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_d_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_d_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu1_i_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu0_i_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_system_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_out_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_dma_uc_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_vpu_l2_in_atrans_unit.attr.attr,
+
+ /* Filtered Event Aliases (RPi 1-4) */
+ &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_icache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_icache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu0_dcache_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_icache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_cpu1_dcache_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma0_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_dma1_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d0_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d1_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_isp_system_l2_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_usb_system_l2_wtrans_unit.attr.attr,
+
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback for event aliases
+ * @kobj: Pointer to sysfs kobject for events directory
+ * @attr: Pointer to sysfs attribute being queried
+ * @unused: Index (unused)
+ *
+ * Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ *
+ * Return: attr->mode (0444) if visible on current SoC, 0 to hide.
+ */
+static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ return attr->mode;
+}
+
+static const struct attribute_group rpi_axi_pmu_events_group = {
+ .name = "events",
+ .attrs = rpi_axi_pmu_events_attrs,
+ .is_visible = rpi_axi_pmu_events_is_visible,
+};
+
+/**
+ * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU core
+ * @dev: Pointer to device structure
+ * @attr: Pointer to device attribute
+ * @buf: Output buffer for cpumask string representation
+ *
+ * Return: Number of bytes written to buffer.
+ */
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+ return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *rpi_axi_pmu_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static const struct attribute_group rpi_axi_pmu_cpumask_group = {
+ .attrs = rpi_axi_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *rpi_axi_pmu_attr_groups[] = {
+ &rpi_axi_pmu_format_group,
+ &rpi_axi_pmu_events_group,
+ &rpi_axi_pmu_cpumask_group,
+ NULL,
+};
+
+/**
+ * rpi_axi_pmu__validate_event() - Validates single event resource capability
+ * @pmu: Pointer to core PMU struct
+ * @fake_hw_events: Temporary array of watcher structures for pre-flight check
+ * @event: Pointer to perf_event being validated
+ *
+ * Return: true if event can be scheduled, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_event(struct pmu *pmu,
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX],
+ struct perf_event *event)
+{
+ enum monitor mon;
+
+ if (is_software_event(event))
+ return true;
+ if (event->pmu != pmu)
+ return false;
+
+ mon = config_to_monitor(event->attr.config);
+ return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0;
+}
+
+/**
+ * rpi_axi_pmu__validate_group() - Validates event group scheduleability
+ * @event: Pointer to leader or sibling perf_event
+ *
+ * Simulates bus watcher allocation across both monitors to guarantee all events
+ * in the group can run simultaneously.
+ *
+ * Return: true if group is valid, false otherwise.
+ */
+static bool rpi_axi_pmu__validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct rpi_axi_hw_events fake_hw_events[MON__MAX];
+
+ rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]);
+ rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]);
+
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader))
+ return false;
+
+ for_each_sibling_event(sibling, leader) {
+ if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling))
+ return false;
+ }
+ return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event);
+}
+
+/**
+ * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event
+ * @event: Pointer to perf_event to initialize
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_event_init(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu;
+ struct device *dev;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ dev = pmu->pmu.dev;
+
+ if (!config_is_valid(pmu, event->attr.config)) {
+ dev_dbg(dev, "Invalid event config\n");
+ return -EINVAL;
+ }
+
+ if (is_sampling_event(event)) {
+ dev_dbg(dev, "Sampling not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu < 0) {
+ dev_dbg(dev, "Per-task data not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (event->cpu != pmu->cpu) {
+ dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu);
+ return -EINVAL;
+ }
+
+ if (!rpi_axi_pmu__validate_group(event)) {
+ dev_dbg(dev, "Invalid event or grouping of events\n");
+ return -EINVAL;
+ }
+ event->hw.idx = -1;
+ return 0;
+}
+
+/**
+ * set_monitor_control() - Writes to global monitor control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection (MON__SYSTEM or MON__VPU)
+ * @set: Control bitmask to write
+ */
+static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set)
+{
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set monitor control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + GEN_CTRL);
+ }
+}
+
+static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+ return BW0_CTRL + idx * BW_PITCH;
+}
+
+/**
+ * set_bus_watcher_control() - Writes to a specific bus watcher control register
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @set: Control bitmask to write
+ */
+static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {pmu->monitor[mon].mailbox + watcher, 1, set};
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_SET_PERIPH_REG,
+ tmp, sizeof(tmp));
+ if (err < 0 || tmp[1] != 1)
+ dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n");
+ } else {
+ lockdep_assert_held(&pmu->lock);
+ writel(set, pmu->monitor[mon].base_address + watcher);
+ }
+}
+
+/**
+ * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @bus: Monitored AXI bus index
+ * @filter: AXI master ID filter
+ */
+static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon,
+ int idx, int bus, int filter)
+{
+ int bus_control;
+
+ if (pmu->monitor[mon].hw_events.enabled[idx])
+ return;
+
+ bus_control = BW_CTRL_ENABLE_BIT | (bus & 0x3F);
+ if (filter) {
+ bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT;
+ bus_control |= (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT;
+ }
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+ set_bus_watcher_control(pmu, mon, idx, bus_control);
+
+ if (!pmu->monitor[mon].hw_events.monitor_running) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT);
+ }
+}
+
+/**
+ * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ */
+static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx)
+{
+
+ set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT);
+}
+
+static int counter_offset(enum counter counter)
+{
+ switch (counter) {
+ case CNT__ATRANS: return BW_ATRANS_OFFSET;
+ case CNT__ATWAIT: return BW_ATWAIT_OFFSET;
+ case CNT__WTRANS: return BW_WTRANS_OFFSET;
+ case CNT__WTWAIT: return BW_WTWAIT_OFFSET;
+ case CNT__RTRANS: return BW_RTRANS_OFFSET;
+ case CNT__RTWAIT: return BW_RTWAIT_OFFSET;
+ default: return 0;
+ }
+}
+
+/**
+ * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher
+ * @pmu: Pointer to PMU context
+ * @mon: Monitor selection
+ * @idx: Watcher index (0..2)
+ * @counter: Metric counter type (enum counter, 0..8)
+ *
+ * Return: 32-bit raw hardware counter value.
+ */
+static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx,
+ enum counter counter)
+{
+ int watcher = watcher_offset(pmu, mon, idx);
+ int offset = counter_offset(counter);
+ u32 ret;
+
+ if (!pmu->monitor[mon].hw_events.enabled[idx])
+ return 0;
+
+ if (pmu->monitor[mon].use_mailbox_interface) {
+ u32 tmp[3] = {
+ pmu->monitor[mon].mailbox + watcher + offset,
+ 1, -1
+ };
+ int err;
+
+ might_sleep();
+ lockdep_assert_held(&pmu->vpu_mutex);
+ if (WARN_ON_ONCE(in_interrupt() || irqs_disabled()))
+ return -1;
+
+ err = rpi_firmware_property(pmu->firmware,
+ RPI_FIRMWARE_GET_PERIPH_REG,
+ tmp, sizeof(tmp));
+
+ if (err < 0 || tmp[1] != 1) {
+ dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n");
+ /*
+ * Return U32_MAX on IPC failure to indicate invalid read.
+ * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF,
+ * so U32_MAX is unambiguously an error state.
+ */
+ return U32_MAX;
+ }
+ ret = tmp[2] & 0x7FFFFFFF;
+ } else {
+ void __iomem *addr = pmu->monitor[mon].base_address + watcher + offset;
+
+ lockdep_assert_held(&pmu->lock);
+ ret = readl(addr) & 0x7FFFFFFF;
+ }
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu_read() - Perf callback to update event counter value
+ * @event: Pointer to perf_event being read
+ *
+ * Thread-safe counter update implementation:
+ * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock.
+ * - VPU Monitor (MON__VPU): Async background polling in process context (vpu_work).
+ * Direct read() syscalls return cached count immediately without blocking.
+ */
+static void rpi_axi_pmu_read(struct perf_event *event)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ u64 prev_count, new_count;
+ unsigned long flags;
+ u32 delta;
+
+ /* Mailbox VPU counters are polled asynchronously in background vpu_work.
+ * MMIO monitors (System) are read synchronously.
+ */
+ if (event->hw.idx < 0)
+ return;
+
+ if (pmu->monitor[mon].use_mailbox_interface)
+ return;
+
+ raw_spin_lock_irqsave(&pmu->lock, flags);
+ if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] ||
+ (event->hw.state & PERF_HES_STOPPED)) {
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter);
+ if (new_count == U32_MAX) {
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return;
+ }
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+}
+
+/**
+ * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads
+ * @work: Pointer to work_struct inside struct rpi_axi_pmu
+ *
+ * Runs in kernel process context where sleeping is allowed. Reads fresh VPU counter values over
+ * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event count safely.
+ * ABA pointer recycling races are prevented by validating per-slot sequence counters (event_gen).
+ * Multiplexing counter rotation baselines are established using the PERF_HES_UPTODATE flag.
+ */
+static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work)
+{
+ struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work);
+
+ might_sleep();
+ mutex_lock(&pmu->vpu_mutex);
+ raw_spin_lock_irq(&pmu->lock);
+
+ for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) {
+ if (pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx]) {
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = false;
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx);
+ }
+ }
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 gen;
+ int idx;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__VPU)
+ continue;
+
+ gen = pmu->event_gen[i];
+ counter = config_to_counter(event->attr.config);
+ idx = event->hw.idx;
+
+ /* If VPU bus watcher is not enabled on hardware, enable it in process context */
+ if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) {
+ int bus = pmu->monitor[MON__VPU].hw_events.monitored_bus[idx];
+ int filter = pmu->monitor[MON__VPU].hw_events.filter[idx];
+
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = true;
+ }
+
+ /* Drop spinlock during Mailbox IPC read only if using mailbox interface */
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter);
+ }
+
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count == U32_MAX)
+ continue;
+
+ /*
+ * Verify event pointer and generation sequence counter match
+ * (ABA pointer recycling race prevention).
+ */
+ if (pmu->events[i] == event &&
+ pmu->event_gen[i] == gen &&
+ !(event->hw.state & PERF_HES_STOPPED)) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = true;
+ if (!(event->hw.state & PERF_HES_UPTODATE)) {
+ /* Initial baseline read for newly started/rotated VPU event */
+ local64_set(&event->hw.prev_count, new_count);
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else {
+ prev_count = local64_read(&event->hw.prev_count);
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+ }
+
+ if (pmu->monitor[MON__VPU].hw_events.num_monitored == 0 &&
+ pmu->monitor[MON__VPU].hw_events.monitor_running) {
+ if (pmu->monitor[MON__VPU].use_mailbox_interface) {
+ raw_spin_unlock_irq(&pmu->lock);
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ raw_spin_lock_irq(&pmu->lock);
+ } else {
+ set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT);
+ }
+ pmu->monitor[MON__VPU].hw_events.monitor_running = false;
+ }
+
+ raw_spin_unlock_irq(&pmu->lock);
+ mutex_unlock(&pmu->vpu_mutex);
+}
+
+/**
+ * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling
+ * @timer: Pointer to hrtimer structure
+ *
+ * Runs in hard IRQ / atomic context.
+ * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast MMIO.
+ * Schedules background work (vpu_work) if active_vpu_events > 0.
+ *
+ * Return: HRTIMER_RESTART.
+ */
+static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer)
+{
+ struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer);
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&pmu->lock, flags);
+ if (pmu->active_events == 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_NORESTART;
+ }
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ struct perf_event *event = pmu->events[i];
+ enum counter counter;
+ u64 prev_count, new_count;
+ u32 delta;
+
+ if (!event || (event->hw.state & PERF_HES_STOPPED) ||
+ config_to_monitor(event->attr.config) != MON__SYSTEM)
+ continue;
+
+ counter = config_to_counter(event->attr.config);
+ prev_count = local64_read(&event->hw.prev_count);
+ new_count = rpi_axi_pmu_read_counter(pmu, MON__SYSTEM,
+ event->hw.idx, counter);
+ /*
+ * U32_MAX indicates a hardware or IPC read failure. Ignore the update
+ * to prevent spurious artificial counter spikes from underflows.
+ */
+ if (new_count != U32_MAX) {
+ local64_set(&event->hw.prev_count, new_count);
+ delta = (new_count - prev_count) & 0x7FFFFFFF;
+ local64_add(delta, &event->count);
+ }
+ }
+
+ if (pmu->active_vpu_events > 0)
+ schedule_work(&pmu->vpu_work);
+
+ hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL);
+ raw_spin_unlock_irqrestore(&pmu->lock, flags);
+ return HRTIMER_RESTART;
+}
+
+/**
+ * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher
+ * @event: Pointer to perf_event being started
+ * @flags: Start control flags
+ */
+static void rpi_axi_pmu_start(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ enum counter counter = config_to_counter(event->attr.config);
+ unsigned long spinflags;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ event->hw.state = 0;
+ if (mon == MON__SYSTEM) {
+ int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx];
+ int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx];
+
+ rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter);
+ pmu->monitor[mon].hw_events.enabled[event->hw.idx] = true;
+ pmu->monitor[mon].hw_events.monitor_running = true;
+ local64_set(&event->hw.prev_count,
+ rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter));
+ event->hw.state |= PERF_HES_UPTODATE;
+ } else if (mon == MON__VPU) {
+ schedule_work(&pmu->vpu_work);
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU
+ * @event: Pointer to perf_event being added
+ * @flags: Add control flags (e.g. PERF_EF_START)
+ *
+ * Return: 0 on success, -EAGAIN if no watcher slot is available.
+ */
+static int rpi_axi_pmu_add(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events;
+ unsigned long spinflags;
+ int idx, slot = -1;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event);
+ if (idx < 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -EAGAIN;
+ }
+
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (!pmu->events[i]) {
+ slot = i;
+ pmu->events[i] = event;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (slot < 0) {
+ hw_events->refcount[idx]--;
+ if (hw_events->refcount[idx] == 0) {
+ hw_events->monitored_bus[idx] = -1;
+ hw_events->filter[idx] = BCM2835_FLT__NONE;
+ hw_events->num_monitored--;
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return -ENOSPC;
+ }
+
+ pmu->active_events++;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events++;
+
+ if (pmu->active_events == 1)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+
+ event->hw.idx = idx;
+ event->hw.state = PERF_HES_STOPPED;
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ if (flags & PERF_EF_START)
+ rpi_axi_pmu_start(event, /*flags=*/0);
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu_stop() - Stops counter monitoring for an event
+ * @event: Pointer to perf_event being stopped
+ * @flags: Stop control flags (e.g. PERF_EF_UPDATE)
+ */
+static void rpi_axi_pmu_stop(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ unsigned long spinflags;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ if (flags & PERF_EF_UPDATE)
+ rpi_axi_pmu_read(event);
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (flags & PERF_EF_UPDATE)
+ event->hw.state |= PERF_HES_UPTODATE;
+ event->hw.state |= PERF_HES_STOPPED;
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU
+ * @event: Pointer to perf_event being removed
+ * @flags: Delete control flags
+ */
+static void rpi_axi_pmu_del(struct perf_event *event, int flags)
+{
+ struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu);
+ enum monitor mon = config_to_monitor(event->attr.config);
+ unsigned long spinflags;
+ int idx = event->hw.idx;
+
+ if (idx < 0)
+ return;
+
+ rpi_axi_pmu_stop(event, PERF_EF_UPDATE);
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) {
+ if (pmu->events[i] == event) {
+ pmu->events[i] = NULL;
+ pmu->event_gen[i]++;
+ break;
+ }
+ }
+
+ if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) {
+ pmu->monitor[mon].hw_events.refcount[idx]--;
+ if (pmu->monitor[mon].hw_events.refcount[idx] == 0) {
+ pmu->monitor[mon].hw_events.monitored_bus[idx] = -1;
+ pmu->monitor[mon].hw_events.filter[idx] = BCM2835_FLT__NONE;
+ pmu->monitor[mon].hw_events.num_monitored--;
+ if (mon == MON__SYSTEM) {
+ pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] = false;
+ rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
+ if (pmu->monitor[mon].hw_events.num_monitored == 0) {
+ set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
+ pmu->monitor[mon].hw_events.monitor_running = false;
+ }
+ } else if (mon == MON__VPU) {
+ pmu->monitor[MON__VPU].hw_events.enabled[idx] = false;
+ pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = true;
+ schedule_work(&pmu->vpu_work);
+ }
+ }
+ }
+
+ event->hw.idx = -1;
+ pmu->active_events--;
+ if (mon == MON__VPU)
+ pmu->active_vpu_events--;
+
+ if (pmu->active_events == 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ return;
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+}
+
+/**
+ * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline
+ * @cpu: CPU core number being taken offline
+ * @node: Pointer to hlist_node inside struct rpi_axi_pmu
+ *
+ * Return: 0.
+ */
+static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+ struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node);
+ unsigned long spinflags;
+ unsigned int target;
+
+ if (!pmu->is_registered)
+ return 0;
+
+ if (cpu != pmu->cpu)
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+ if (target >= nr_cpu_ids)
+ return 0;
+
+ perf_pmu_migrate_context(&pmu->pmu, cpu, target);
+ pmu->cpu = target;
+
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0) {
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+ hrtimer_cancel(&pmu->hrtimer);
+ raw_spin_lock_irqsave(&pmu->lock, spinflags);
+ if (pmu->active_events > 0)
+ hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL,
+ HRTIMER_MODE_REL_SOFT);
+ }
+ raw_spin_unlock_irqrestore(&pmu->lock, spinflags);
+
+ return 0;
+}
+
+/**
+ * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe
+ * @pmu: Pointer to PMU context
+ * @pdev: Platform device pointer
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *fw_node;
+ int ret;
+
+ raw_spin_lock_init(&pmu->lock);
+ mutex_init(&pmu->vpu_mutex);
+
+ pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev);
+
+ pmu->pmu = (struct pmu) {
+ .module = THIS_MODULE,
+ .attr_groups = rpi_axi_pmu_attr_groups,
+ .task_ctx_nr = perf_invalid_context,
+ .event_init = rpi_axi_pmu_event_init,
+ .add = rpi_axi_pmu_add,
+ .del = rpi_axi_pmu_del,
+ .start = rpi_axi_pmu_start,
+ .stop = rpi_axi_pmu_stop,
+ .read = rpi_axi_pmu_read,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
+ };
+ pmu->pdev = pdev;
+ pmu->cpu = cpumask_first(cpu_online_mask);
+ hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler);
+
+ if (pmu->chip == CHIP_BCM2835) {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ fw_node = of_parse_phandle(dev->of_node, "firmware", 0);
+ if (fw_node) {
+ pmu->firmware = rpi_firmware_get(fw_node);
+ of_node_put(fw_node);
+ }
+ if (!pmu->firmware)
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ else
+ pmu->monitor[MON__VPU].use_mailbox_interface = true;
+ } else {
+ pmu->monitor[MON__SYSTEM].use_mailbox_interface = false;
+ pmu->monitor[MON__VPU].use_mailbox_interface = false;
+ }
+
+ for (int i = 0; i < MON__MAX; i++) {
+ rpi_axi_hw_events__init(&pmu->monitor[i].hw_events);
+
+ if (pmu->monitor[i].use_mailbox_interface) {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource) {
+ dev_err(dev, "Error reading mailbox resource %d\n", i);
+ ret = -EINVAL;
+ goto err_firmware_put;
+ }
+ pmu->monitor[i].mailbox = (u32)resource->start;
+ } else {
+ struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i);
+
+ if (!resource)
+ continue;
+
+ pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource);
+ if (IS_ERR(pmu->monitor[i].base_address)) {
+ ret = PTR_ERR(pmu->monitor[i].base_address);
+ dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret);
+ goto err_firmware_put;
+ }
+ }
+ }
+
+ ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ if (ret) {
+ dev_err(dev, "Failed to add cpuhp instance %d\n", ret);
+ goto err_teardown;
+ }
+
+ ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1);
+ if (ret) {
+ dev_err(dev, "PMU register failed %d\n", ret);
+ goto err_cpuhp_remove;
+ }
+
+ pmu->is_registered = true;
+ return 0;
+
+err_cpuhp_remove:
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+err_teardown:
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+err_firmware_put:
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+ return ret;
+}
+
+/**
+ * rpi_axi_pmu__exit() - Internal PMU teardown called during remove
+ * @pmu: Pointer to PMU context
+ */
+static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
+{
+ cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
+ perf_pmu_unregister(&pmu->pmu);
+ hrtimer_cancel(&pmu->hrtimer);
+ cancel_work_sync(&pmu->vpu_work);
+ if (pmu->firmware)
+ rpi_firmware_put(pmu->firmware);
+}
+
+/* --- DRIVER ENTRY POINTS ----------------------------------------- */
+
+/**
+ * rpi_axi_pmu_probe() - Platform driver probe entry point
+ * @pdev: Pointer to platform_device
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int rpi_axi_pmu_probe(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL);
+ if (!pmu)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, pmu);
+ return rpi_axi_pmu__init(pmu, pdev);
+}
+
+/**
+ * rpi_axi_pmu_remove() - Platform driver remove entry point
+ * @pdev: Pointer to platform_device
+ */
+static void rpi_axi_pmu_remove(struct platform_device *pdev)
+{
+ struct rpi_axi_pmu *pmu;
+
+ pmu = platform_get_drvdata(pdev);
+ rpi_axi_pmu__exit(pmu);
+}
+
+/* Devices matching this driver in Device Tree */
+static const struct of_device_id rpi_axi_pmu_match[] = {
+ {
+ .compatible = "brcm,bcm2835-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ {
+ .compatible = "brcm,bcm2711-axiperf",
+ .data = (void *)CHIP_BCM2835,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
+
+static struct platform_driver rpi_axi_pmu_driver = {
+ .probe = rpi_axi_pmu_probe,
+ .remove = rpi_axi_pmu_remove,
+ .driver = {
+ .name = PMU_NAME,
+ .of_match_table = of_match_ptr(rpi_axi_pmu_match),
+ .suppress_bind_attrs = true,
+ },
+};
+
+/**
+ * rpi_axi_pmu_driver_init() - Module initialization entry point
+ *
+ * Registers the dynamic CPU hotplug state and the platform driver.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int __init rpi_axi_pmu_driver_init(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online",
+ NULL, rpi_axi_pmu_offline_cpu);
+ if (ret < 0)
+ return ret;
+
+ rpi_axi_pmu_cpuhp_state = ret;
+
+ ret = platform_driver_register(&rpi_axi_pmu_driver);
+ if (ret)
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+
+ return ret;
+}
+module_init(rpi_axi_pmu_driver_init);
+
+/**
+ * rpi_axi_pmu_driver_exit() - Module cleanup entry point
+ *
+ * Unregisters the platform driver and CPU hotplug state.
+ */
+static void __exit rpi_axi_pmu_driver_exit(void)
+{
+ platform_driver_unregister(&rpi_axi_pmu_driver);
+ cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state);
+}
+module_exit(rpi_axi_pmu_driver_exit);
+
+MODULE_LICENSE("GPL");
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
` (3 preceding siblings ...)
(?)
@ 2026-08-13 8:03 ` Ian Rogers
-1 siblings, 0 replies; 59+ messages in thread
From: Ian Rogers @ 2026-08-13 8:03 UTC (permalink / raw)
To: irogers, linux-perf-users, linux-rpi-kernel
Cc: linux-arm-kernel, linux-kernel, mark.rutland, will
This patch introduces AXI PMU support for the Broadcom BCM2712 SoC used
in the Raspberry Pi 5. It expands the driver with BCM2712-specific
buses, hardware filters, and sysfs events, while ensuring compatibility
with prior generations.
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/perf/rpi_axi_pmu.c | 429 ++++++++++++++++++++++++++++++++++++-
1 file changed, 421 insertions(+), 8 deletions(-)
diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
index 676e8f873163..f6c5c6788a08 100644
--- a/drivers/perf/rpi_axi_pmu.c
+++ b/drivers/perf/rpi_axi_pmu.c
@@ -6,6 +6,7 @@
* This driver exposes the performance monitoring hardware on Raspberry Pi
* System-on-Chips to the Linux perf subsystem:
* - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711).
+ * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712).
*
* Architecture Overview:
* ----------------------
@@ -19,6 +20,7 @@
* Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.).
* Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG).
* Read latency: ~10-100 microseconds (IPC over VPU mailbox).
+ * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and MMIO mapping.
*
* Synchronization & Concurrency Model:
* ------------------------------------
@@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *timer,
/**
* enum rpi_axi_chip - Supported Broadcom SoC generations
* @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, Zero/W)
+ * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5)
*/
enum rpi_axi_chip {
CHIP_BCM2835 = 0,
+ CHIP_BCM2712,
};
enum monitor {
@@ -126,6 +130,43 @@ enum bcm2835_system_bus {
BCM2835_SB__MAX
};
+/**
+ * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2712 (RPi 5)
+ * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus
+ * @BCM2712_SB__TRANS: Transposer engine bus
+ * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus
+ * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus
+ * @BCM2712_SB__DMA_UC: DMA Uncached memory bus
+ * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus
+ * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, Ethernet, I/O)
+ * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus
+ * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (DSU) L3 bus
+ * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engine bus
+ * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus
+ * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus
+ * @BCM2712_SB__PERIPHERAL: System peripherals bus
+ * @BCM2712_SB__CPU_UC: CPU Uncached memory bus
+ * @BCM2712_SB__CPU_L2: CPU L2 cache bus
+ * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712
+ */
+enum bcm2712_system_bus {
+ BCM2712_SB__DMA_L2 = 0,
+ BCM2712_SB__TRANS,
+ BCM2712_SB__JPEG,
+ BCM2712_SB__SYSTEM_UC,
+ BCM2712_SB__DMA_UC,
+ BCM2712_SB__SYSTEM_L2,
+ BCM2712_SB__PCIE_RP1,
+ BCM2712_SB__HEVC_DEC,
+ BCM2712_SB__A76_DSU_L3,
+ BCM2712_SB__HVS,
+ BCM2712_SB__V3D7,
+ BCM2712_SB__ISP,
+ BCM2712_SB__PERIPHERAL,
+ BCM2712_SB__CPU_UC,
+ BCM2712_SB__CPU_L2,
+ BCM2712_SB__MAX
+};
/**
* enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (RPi 1-4)
@@ -259,6 +300,93 @@ enum bcm2835_filter {
BCM2835_FLT__MAX
};
+/**
+ * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5)
+ * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on bus)
+ * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID
+ * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID
+ * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID
+ * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID
+ * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID
+ * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID
+ * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID
+ * @BCM2712_FLT__DMA2: DMA2 master ID
+ * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID
+ * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID
+ * @BCM2712_FLT__DMA0: DMA0 master ID
+ * @BCM2712_FLT__DMA1: DMA1 master ID
+ * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID
+ * @BCM2712_FLT__BBSI: BBSI master ID
+ * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID
+ * @BCM2712_FLT__PCIE1: PCIe 1 master ID
+ * @BCM2712_FLT__PCIE2: PCIe 2 master ID
+ * @BCM2712_FLT__UMR: UMR master ID
+ * @BCM2712_FLT__SAGE: SAGE master ID
+ * @BCM2712_FLT__HVDP: HVDP master ID
+ * @BCM2712_FLT__BSP: BSP master ID
+ * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID
+ * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID
+ * @BCM2712_FLT__MOP0: MOP0 master ID
+ * @BCM2712_FLT__MOP1: MOP1 master ID
+ * @BCM2712_FLT__MBVN: MBVN master ID
+ * @BCM2712_FLT__DSI: DSI Display Interface master ID
+ * @BCM2712_FLT__XPT: XPT master ID
+ * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID
+ * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID
+ * @BCM2712_FLT__USB: USB Controller master ID
+ * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712
+ */
+enum bcm2712_filter {
+ BCM2712_FLT__NONE = 0,
+ BCM2712_FLT__VPU_UC0 = 1,
+ BCM2712_FLT__VPU_IC0 = 2,
+ BCM2712_FLT__VPU_DC0 = 3,
+ BCM2712_FLT__VPU_UC1 = 4,
+ BCM2712_FLT__VPU_IC1 = 5,
+ BCM2712_FLT__VPU_DC1 = 6,
+ BCM2712_FLT__VPU_L2 = 7,
+ BCM2712_FLT__DMA2 = 8,
+ BCM2712_FLT__VPU_DEBUG = 9,
+ BCM2712_FLT__ARM = 10,
+ BCM2712_FLT__DMA0 = 11,
+ BCM2712_FLT__DMA1 = 12,
+ BCM2712_FLT__RAAGA = 13,
+ BCM2712_FLT__BBSI = 14,
+ BCM2712_FLT__PCIE0 = 15,
+ BCM2712_FLT__PCIE1 = 16,
+ BCM2712_FLT__PCIE2 = 17,
+ BCM2712_FLT__UMR = 18,
+ BCM2712_FLT__SAGE = 19,
+ BCM2712_FLT__HVDP = 20,
+ BCM2712_FLT__BSP = 21,
+ BCM2712_FLT__HVS = 22,
+ BCM2712_FLT__HVS_WMK = 23,
+ BCM2712_FLT__MOP0 = 24,
+ BCM2712_FLT__MOP1 = 25,
+ BCM2712_FLT__MBVN = 26,
+ BCM2712_FLT__DSI = 27,
+ BCM2712_FLT__XPT = 28,
+ BCM2712_FLT__EMMC0 = 29,
+ BCM2712_FLT__GENET = 30,
+ BCM2712_FLT__USB = 31,
+ BCM2712_FLT__MAX = 32
+};
+
+/* Compile-time static assertions verifying cross-generation enum equivalence */
+static_assert((int)BCM2835_FLT__NONE == (int)BCM2712_FLT__NONE,
+ "Filter NONE enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_L2 == (int)BCM2712_SB__DMA_L2,
+ "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__TRANS == (int)BCM2712_SB__TRANS,
+ "TRANS bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__JPEG == (int)BCM2712_SB__JPEG,
+ "JPEG bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_UC == (int)BCM2712_SB__SYSTEM_UC,
+ "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__DMA_UC == (int)BCM2712_SB__DMA_UC,
+ "DMA_UC bus enum value mismatch across BCM2835 and BCM2712");
+static_assert((int)BCM2835_SB__SYSTEM_L2 == (int)BCM2712_SB__SYSTEM_L2,
+ "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712");
/* Hardware register offsets & control bitwise constants */
#define GEN_CTRL 0x00
@@ -445,7 +573,7 @@ static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_e
* struct rpi_axi_pmu - Root PMU driver context
* @pmu: Core Linux perf PMU structure
* @pdev: Owning platform_device pointer
- * @chip: Detected Broadcom SoC generation
+ * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712)
* @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711)
* @cpu: CPU core assigned to process uncore PMU events
* @cpuhp_node: Dynamic CPU hotplug instance node
@@ -507,15 +635,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config)
if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address)
return false;
- if (mon == MON__SYSTEM) {
- if (bus >= BCM2835_SB__MAX)
+ if (pmu->chip == CHIP_BCM2712) {
+ if (mon != MON__SYSTEM)
+ return false;
+ if (bus >= BCM2712_SB__MAX)
+ return false;
+ if (filter >= BCM2712_FLT__MAX)
return false;
} else {
- if (bus >= VPU__MAX)
+ if (mon == MON__SYSTEM) {
+ if (bus >= BCM2835_SB__MAX)
+ return false;
+ } else {
+ if (bus >= VPU__MAX)
+ return false;
+ }
+ if (filter >= BCM2835_FLT__MAX)
return false;
}
- if (filter >= BCM2835_FLT__MAX)
- return false;
return counter < CNT__MAX;
}
@@ -541,7 +678,7 @@ static const struct attribute_group rpi_axi_pmu_format_group = {
* Event Attribute Scaling & Unit Definitions:
*
* Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report hardware
- * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat
+ * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, single-beat
* transactions transfer 32 bytes per beat, while contiguous DMA page bursts may
* use 64-byte double-beats. Setting .scale="32" and .unit="Bytes" provides a
* close (~95%) byte throughput approximation in perf stat output (accounting for
@@ -1175,6 +1312,171 @@ PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.scale, rpi_axi_pmu_event_usb_system_l
PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans.unit, rpi_axi_pmu_event_usb_system_l2_wtrans_unit,
"Bytes");
+/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES ---------------------- */
+
+/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=6, BCM2712_SB__PCIE_RP1) */
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans,
+ "monitor=0,bus=6,counter=0");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans.scale, rpi_axi_pmu_event_pcie_rp1_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atrans.unit, rpi_axi_pmu_event_pcie_rp1_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait,
+ "monitor=0,bus=6,counter=1");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans,
+ "monitor=0,bus=6,counter=2");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans.scale, rpi_axi_pmu_event_pcie_rp1_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans.unit, rpi_axi_pmu_event_pcie_rp1_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait,
+ "monitor=0,bus=6,counter=3");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans,
+ "monitor=0,bus=6,counter=4");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans.scale, rpi_axi_pmu_event_pcie_rp1_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans.unit, rpi_axi_pmu_event_pcie_rp1_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait,
+ "monitor=0,bus=6,counter=5");
+
+/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=7, BCM2712_SB__HEVC_DEC) */
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans,
+ "monitor=0,bus=7,counter=0");
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans.scale, rpi_axi_pmu_event_hevc_dec_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_atrans.unit, rpi_axi_pmu_event_hevc_dec_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans,
+ "monitor=0,bus=7,counter=2");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans.scale, rpi_axi_pmu_event_hevc_dec_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_wtrans.unit, rpi_axi_pmu_event_hevc_dec_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans,
+ "monitor=0,bus=7,counter=4");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans.scale, rpi_axi_pmu_event_hevc_dec_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hevc_dec_rtrans.unit, rpi_axi_pmu_event_hevc_dec_rtrans_unit,
+ "Bytes");
+
+/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=8, BCM2712_SB__A76_DSU_L3) */
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atrans,
+ "monitor=0,bus=8,counter=0");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans.scale, rpi_axi_pmu_event_a76_dsu_l3_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans.unit, rpi_axi_pmu_event_a76_dsu_l3_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtrans,
+ "monitor=0,bus=8,counter=2");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans.scale, rpi_axi_pmu_event_a76_dsu_l3_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans.unit, rpi_axi_pmu_event_a76_dsu_l3_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtrans,
+ "monitor=0,bus=8,counter=4");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans.scale, rpi_axi_pmu_event_a76_dsu_l3_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans.unit, rpi_axi_pmu_event_a76_dsu_l3_rtrans_unit,
+ "Bytes");
+
+/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=10, BCM2712_SB__V3D7) */
+PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans,
+ "monitor=0,bus=10,counter=0");
+PMU_EVENT_ATTR_STRING(v3d7_atrans.scale, rpi_axi_pmu_event_v3d7_atrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_atrans.unit, rpi_axi_pmu_event_v3d7_atrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans,
+ "monitor=0,bus=10,counter=2");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans.scale, rpi_axi_pmu_event_v3d7_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_wtrans.unit, rpi_axi_pmu_event_v3d7_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans,
+ "monitor=0,bus=10,counter=4");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans.scale, rpi_axi_pmu_event_v3d7_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(v3d7_rtrans.unit, rpi_axi_pmu_event_v3d7_rtrans_unit,
+ "Bytes");
+
+/*
+ * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events
+ * (bus=14, BCM2712_SB__CPU_L2)
+ */
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712_peripheral_rtrans,
+ "monitor=0,bus=12,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans.scale,
+ rpi_axi_pmu_event_bcm2712_peripheral_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans.unit,
+ rpi_axi_pmu_event_bcm2712_peripheral_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712_peripheral_wtrans,
+ "monitor=0,bus=12,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans.scale,
+ rpi_axi_pmu_event_bcm2712_peripheral_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans.unit,
+ rpi_axi_pmu_event_bcm2712_peripheral_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans,
+ "monitor=0,bus=13,counter=4");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans.scale, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans.unit, rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans,
+ "monitor=0,bus=13,counter=2");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans.scale, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans.unit, rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_unit,
+ "Bytes");
+
+/* BCM2712 Master ID Filtered Events */
+PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans,
+ "monitor=0,bus=14,counter=4,filter=10");
+PMU_EVENT_ATTR_STRING(arm_rtrans.scale, rpi_axi_pmu_event_arm_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(arm_rtrans.unit, rpi_axi_pmu_event_arm_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans,
+ "monitor=0,bus=14,counter=2,filter=10");
+PMU_EVENT_ATTR_STRING(arm_wtrans.scale, rpi_axi_pmu_event_arm_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(arm_wtrans.unit, rpi_axi_pmu_event_arm_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans,
+ "monitor=0,bus=6,counter=4,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans.scale, rpi_axi_pmu_event_pcie0_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie0_rtrans.unit, rpi_axi_pmu_event_pcie0_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans,
+ "monitor=0,bus=6,counter=2,filter=15");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans.scale, rpi_axi_pmu_event_pcie0_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(pcie0_wtrans.unit, rpi_axi_pmu_event_pcie0_wtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_rtrans,
+ "monitor=0,bus=9,counter=4,filter=22");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans.scale, rpi_axi_pmu_event_hvs_filtered_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans.unit, rpi_axi_pmu_event_hvs_filtered_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans,
+ "monitor=0,bus=5,counter=4,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans.scale, rpi_axi_pmu_event_emmc0_rtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(emmc0_rtrans.unit, rpi_axi_pmu_event_emmc0_rtrans_unit,
+ "Bytes");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans,
+ "monitor=0,bus=5,counter=2,filter=29");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans.scale, rpi_axi_pmu_event_emmc0_wtrans_scale,
+ "32");
+PMU_EVENT_ATTR_STRING(emmc0_wtrans.unit, rpi_axi_pmu_event_emmc0_wtrans_unit,
+ "Bytes");
+
static struct attribute *rpi_axi_pmu_events_attrs[] = {
/* System Monitor events */
&rpi_axi_pmu_event_dma_l2_atrans.attr.attr,
@@ -1480,6 +1782,84 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
&rpi_axi_pmu_event_usb_system_l2_wtrans_scale.attr.attr,
&rpi_axi_pmu_event_usb_system_l2_wtrans_unit.attr.attr,
+ /* RPi 5 (BCM2712) Expanded Event Aliases */
+ &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr,
+
+ &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hevc_dec_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_a76_dsu_l3_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_v3d7_atrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_atrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_atrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_v3d7_rtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_peripheral_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans_unit.attr.attr,
+
+ &rpi_axi_pmu_event_arm_rtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_arm_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_arm_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie0_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_pcie0_wtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_hvs_filtered_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_emmc0_rtrans_unit.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans_scale.attr.attr,
+ &rpi_axi_pmu_event_emmc0_wtrans_unit.attr.attr,
NULL,
};
@@ -1490,12 +1870,41 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = {
* @unused: Index (unused)
*
* Dynamically filters sysfs event aliases based on the detected Broadcom SoC generation:
+ * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor=1).
+ * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific events.
*
* Return: attr->mode (0444) if visible on current SoC, 0 to hide.
*/
static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj,
struct attribute *attr, int unused)
{
+ struct device *dev = kobj_to_dev(kobj);
+ struct pmu *pmu = dev_get_drvdata(dev);
+ struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu);
+
+
+ if (rpi_pmu->chip == CHIP_BCM2712) {
+ /*
+ * On RPi 5 (BCM2712), aggressively filter out any legacy events,
+ * including ghost scaling parameters by exclusively matching the
+ * bcm2712 prefix.
+ */
+ if (!strstr(attr->name, "bcm2712_"))
+ return 0;
+ } else {
+ /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */
+ if (strstr(attr->name, "pcie_rp1") ||
+ strstr(attr->name, "a76_") ||
+ strstr(attr->name, "v3d7_") ||
+ strstr(attr->name, "hevc_dec") ||
+ strstr(attr->name, "arm_") ||
+ strstr(attr->name, "pcie0_") ||
+ strstr(attr->name, "hvs_filtered") ||
+ strstr(attr->name, "emmc0_") ||
+ strstr(attr->name, "bcm2712_"))
+ return 0;
+ }
+
return attr->mode;
}
@@ -1826,7 +2235,7 @@ static void rpi_axi_pmu_read(struct perf_event *event)
u32 delta;
/* Mailbox VPU counters are polled asynchronously in background vpu_work.
- * MMIO monitors (System) are read synchronously.
+ * MMIO monitors (System and BCM2712 VPU) are read synchronously.
*/
if (event->hw.idx < 0)
return;
@@ -2395,6 +2804,10 @@ static const struct of_device_id rpi_axi_pmu_match[] = {
.compatible = "brcm,bcm2711-axiperf",
.data = (void *)CHIP_BCM2835,
},
+ {
+ .compatible = "brcm,bcm2712-axiperf",
+ .data = (void *)CHIP_BCM2712,
+ },
{ }
};
MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match);
--
2.55.0.691.gc56d675ccc-goog
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: perf: Add Raspberry Pi BCM2835 AXI PMU driver
2026-08-13 8:03 ` perf: Add Raspberry Pi BCM2835 AXI PMU driver Ian Rogers
@ 2026-08-13 8:13 ` sashiko-bot
0 siblings, 0 replies; 59+ messages in thread
From: sashiko-bot @ 2026-08-13 8:13 UTC (permalink / raw)
To: Ian Rogers; +Cc: linux-perf-users
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Hardware monitors are not disabled on module unload due to cancel_work_sync().
- [Low] Out-of-tree version compatibility macro blocks upstream integration.
--
--- Patch [1]: perf: Add Raspberry Pi BCM2835 AXI PMU driver ---
commit bbd5d9f81c5427301be257b124dcb9150ad69d8f
Author: Ian Rogers <irogers@google.com>
perf: Add Raspberry Pi BCM2835 AXI PMU driver
This commit adds a new performance monitoring driver for the Raspberry Pi
AXI bus (BCM2835/2711), exposing system-level and VideoCore PMU hardware to
the Linux perf subsystem.
> diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c
> new file mode 100644
> index 0000000000000..676e8f873163f
> --- /dev/null
> +++ b/drivers/perf/rpi_axi_pmu.c
> @@ -0,0 +1,2450 @@
> +// SPDX-License-Identifier: GPL-2.0-only
[ ... ]
> +#include <linux/io.h>
> +#include <linux/version.h>
> +
> +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0)
> +static inline void rpi_hrtimer_setup(struct hrtimer *timer,
> + enum hrtimer_restart (*function)(struct hrtimer *),
> + clockid_t clock_id, enum hrtimer_mode mode)
> +{
> + hrtimer_init(timer, clock_id, mode);
> + timer->function = function;
> +}
> +#define hrtimer_setup rpi_hrtimer_setup
> +#endif
[Severity: Low]
This isn't a bug, but does this violate upstream kernel submission
guidelines by retaining out-of-tree backward compatibility code?
Upstream policy generally prohibits including LINUX_VERSION_CODE checks for
backward compatibility in mainline submissions.
[ ... ]
> +static void rpi_axi_pmu_del(struct perf_event *event, int flags)
> +{
[ ... ]
> + if (mon == MON__SYSTEM) {
> + pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] = false;
> + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx);
> + if (pmu->monitor[mon].hw_events.num_monitored == 0) {
> + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT);
> + pmu->monitor[mon].hw_events.monitor_running = false;
> + }
> + } else if (mon == MON__VPU) {
> + pmu->monitor[MON__VPU].hw_events.enabled[idx] = false;
> + pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] = true;
> + schedule_work(&pmu->vpu_work);
> + }
> + }
> + }
[ ... ]
> +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu)
> +{
> + cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node);
> + perf_pmu_unregister(&pmu->pmu);
> + hrtimer_cancel(&pmu->hrtimer);
> + cancel_work_sync(&pmu->vpu_work);
[Severity: Medium]
Will this cancel_work_sync() drop the deferred hardware disable commands
scheduled by rpi_axi_pmu_del() during perf_pmu_unregister()?
If the module is unloaded while VPU perf events are active,
perf_pmu_unregister() detaches events and calls rpi_axi_pmu_del(). For VPU
events, this schedules vpu_work to disable the hardware via Mailbox IPC.
Since cancel_work_sync() drops the pending work instead of executing it,
the VideoCore VPU monitor might be left running indefinitely.
Would flush_work() be more appropriate here to ensure the hardware is properly
disabled?
> + if (pmu->firmware)
> + rpi_firmware_put(pmu->firmware);
> +}
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260813080314.2836027-1-irogers@google.com?part=1
^ permalink raw reply [flat|nested] 59+ messages in thread
end of thread, other threads:[~2026-08-13 8:13 UTC | newest]
Thread overview: 59+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-11 8:38 [PATCH v1 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-11 8:38 ` [PATCH v1 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-11 14:32 ` Uwe Kleine-König
2026-08-11 8:38 ` [PATCH v1 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-11 23:51 ` [PATCH v2 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-11 23:51 ` [PATCH v2 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 0:04 ` sashiko-bot
2026-08-11 23:51 ` [PATCH v2 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 0:05 ` sashiko-bot
2026-08-12 0:27 ` [PATCH v3 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 0:27 ` [PATCH v3 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 0:40 ` sashiko-bot
2026-08-12 0:27 ` [PATCH v3 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 0:40 ` sashiko-bot
2026-08-12 5:24 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 5:24 ` [PATCH v4 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 5:36 ` sashiko-bot
2026-08-12 5:24 ` [PATCH v4 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 5:34 ` sashiko-bot
2026-08-12 8:27 ` [PATCH v4 0/2] perf: Add Raspberry Pi AXI PMU driver Will Deacon
2026-08-12 13:38 ` Ian Rogers
2026-08-12 16:26 ` Uwe Kleine-König
2026-08-12 21:37 ` Ian Rogers
2026-08-13 6:59 ` Uwe Kleine-König
2026-08-12 23:01 ` [PATCH v5 " Ian Rogers
2026-08-12 23:01 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 23:15 ` sashiko-bot
2026-08-12 23:01 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 23:18 ` sashiko-bot
2026-08-12 23:07 ` [PATCH v5 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-12 23:07 ` Ian Rogers
2026-08-12 23:07 ` [PATCH v5 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-12 23:25 ` sashiko-bot
2026-08-13 0:06 ` Florian Fainelli
2026-08-12 23:08 ` [PATCH v5 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-12 23:24 ` sashiko-bot
2026-08-13 4:35 ` [PATCH v6 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 4:35 ` [PATCH v6 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-13 4:48 ` sashiko-bot
2026-08-13 4:35 ` [PATCH v6 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 4:43 ` sashiko-bot
2026-08-13 5:24 ` [PATCH v7 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 5:24 ` [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-13 5:35 ` sashiko-bot
2026-08-13 5:24 ` [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 5:46 ` sashiko-bot
2026-08-13 6:42 ` [PATCH v8 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 6:42 ` [PATCH v8 1/2] perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-13 6:51 ` sashiko-bot
2026-08-13 6:42 ` [PATCH v8 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 6:52 ` sashiko-bot
2026-08-13 7:30 ` [PATCH v9 0/2] perf: Add Raspberry Pi AXI PMU driver Ian Rogers
2026-08-13 8:03 ` [PATCH v10 " Ian Rogers
2026-08-13 7:30 ` perf: Add Raspberry Pi BCM2835 " Ian Rogers
2026-08-13 7:40 ` sashiko-bot
2026-08-13 7:30 ` perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
2026-08-13 8:03 ` perf: Add Raspberry Pi BCM2835 AXI PMU driver Ian Rogers
2026-08-13 8:13 ` sashiko-bot
2026-08-13 8:03 ` perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support Ian Rogers
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.