From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=74.125.82.72 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1793663003; b=umc5xoWp6PixkuXZo9K66Z1v92qJufLueXz3PyAljylOR0YVpvd4FkQr3rpn5+J8QoPY Z72ALFFlETM7J10ldPkU8f2jMO2f7eJB4cv6zrXvg6EdjQQYnYN2n1DVFI51H4+x/wP6M Le5ngdIPdcffnsoH/WwoLgiEaVt0Y2Zy5jEghTTt7Wl87al0bB+Yrp7kYF9jgADXfqMW2 ZxLsbYzcvn+lgrpTfrbCGx2m4K2XXXOr84UW+0Q6kQSRGAyF4IxL7XShjwVqksCJiD+VB nY57X4CkSjI2w6CpKcMmV6Us09S7J4HxzeCPOPwiVJOYOGdAGiFs7t2XBHlSy9gYKUQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1793663003; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Received:Date:In-Reply-To: Mime-Version:References:X-Mailer:Message-ID:Subject:From:To:Cc: Content-Type; bh=x7zkEb9CulD+P843VvGXhJhWJgfaplTc5rpEZaq6o80=; b=Bz/EC+WgWfsGfk/oW0z9RPSLW8S989oGrVZLr+BGCTzhrWvpt7My+NzBSP2+if4xrdug AGBmLqlOrRPi6YXoB0vm+1iEB6D3mahbO1DwcD8qdEjKUMKCTasxKEu9v+F+c7HK09YIL xqTkIshKMjy47jqUK2aG1OrZSrhyvoWdxXDnQQtopvV6MUhPVQ5qhjzyUw6CjL7oACHSh 4eELNpFADszN2EFafDyjo/WoYaiEZTuGytDolyuQv/VIWVb4Bu9SGrzyH9DIPBaTPeK4N BST1esBLHGPLtoRQEzUyRUVeHy7itEsLte6Bhi4UPfvBZ/0MmzvPRiMoJmWu6Z14QYw== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=cr4QJFaZ; arc=none smtp.remote-ip=74.125.82.72 Received: from mail-dl1-f72.google.com (mail-dl1-f72.google.com [74.125.82.72]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAC90606D812 for ; Mon, 2 Nov 2026 23:43:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org CAC90606D812 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=cr4QJFaZ Received: by mail-dl1-f72.google.com with SMTP id a92af1059eb24-14373747468so4435257c88.1 for ; Mon, 02 Nov 2026 15:43:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20260707; t=1793663000; x=1794267800; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=x7zkEb9CulD+P843VvGXhJhWJgfaplTc5rpEZaq6o80=; b=cr4QJFaZZLIHuGlgwlsPFrWiAyEY5j/c9waNE5g54AfnP8BlI1uwrJExnpRH6ydy6H xeMM00FOw4Ut6Hs4DMAoDaerS9GF2j2K2wW7aAHhoEG8Vjq38ucAE4vQB8fWBDmE5cpB UvqTEEZgNsk1kdnr45DjPiVHW3s0CKNYEb1Ck9AxSWFqZfsTPVhcttU3a+E7hQ4Ek+SR kZZXTFDDEMPOEoqOtHIubAIiRXKPo2K8/Vr5y3UK+kpaJEWzaO+aIY5aWxd9FRCfCjLG ZB/Zk65R2RLyuB9hk+PcZHklVYieqcNXwDhVy5uM4dC1R82f6n7qVkmmFQYQuUIeiCoZ Webw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793663000; x=1794267800; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=x7zkEb9CulD+P843VvGXhJhWJgfaplTc5rpEZaq6o80=; b=I4jr1gz/PQrVnIWLXq9m8f+eeL9NqYV888J9iH+STB6oJ9cbMwOyoO1420HEkULLUw 9u6aMq7jxB2fhe7xxER0H9lmkdFS/lGjmVMuSZLwa0/lPKdi2Y4114zVkm7z8Yp9LgnB 6F+sHipIOzsCyfxOtqp9a88GI4pxsKWaT/KLCRN8MHgMJdstsr81vpsxavZrCibzwReg A+DU0j8GynSgimk+ncxHW6OAkzpdZN00BYF4DzkVbzN+aLt0IAJPKvNE5mBDB1S8YiU2 84tVUr/xrnsvYWSDV9eXyHbhnLRWbfLxK5ormOOMrD3kQCh9+dewXggTNBF933LYeC/W ossw== X-Forwarded-Encrypted: i=1; AKwUvBztZg23di7Z+SCnvFkdOx2LX5ShycXI1kxYm5xb1cSonKO1rpMTN2rcjw3LjOLr6zaVdR33J6ZPX+8=@vger.kernel.org X-Gm-Message-State: AFq9FYKx85HkL8nciUL18981xj+JPYAHaDEO9VCEdJChXERsNbh/G7ou LdPbd7xQAFWCmakrfNmxHmB6FBqEltCUCrosG+8VbZT6b5PLjqMwT+X7VN5bCiFIFN/XvnwrQFv U8yWz4fqMvg== X-Received: from dlam3-n1.prod.google.com ([2002:a05:701b:2083:10b0:19c:1243:1e3d]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:701b:21db:b0:1b1:251f:d732 with SMTP id a92af1059eb24-1b23f20d4b8mr357307c88.31.1793662998713; Mon, 02 Nov 2026 15:43:18 -0800 (PST) Date: Mon, 2 Nov 2026 15:43:00 -0800 In-Reply-To: <20261102234301.2242242-1-irogers@google.com> X-Mailing-List: linux-doc@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 References: <20261102234301.2242242-1-irogers@google.com> X-Mailer: git-send-email 2.56.0.385.gac671c819a-goog Message-ID: <20261102234301.2242242-3-irogers@google.com> Subject: [PATCH v15 2/3] perf: Add Raspberry Pi BCM2835 AXI PMU driver From: Ian Rogers To: Will Deacon , Mark Rutland , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Florian Fainelli , Dom Cobley Cc: Jonathan Corbet , "=?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?=" , Stefan Wahren , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, linux-perf-users@vger.kernel.org, linux-rpi-kernel@lists.infradead.org, linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Ian Rogers Content-Type: text/plain; charset="UTF-8" Add a PMU driver for the Raspberry Pi AXI performance monitors. It provides access to the System and VideoCore performance counters on Broadcom BCM2835 and BCM2711 platforms natively through the perf subsystem. The driver implements bus-watcher refcounting, group validation, CPU hotplug migration, and sysfs event aliases mapping hardware-specific events to AXI pipelines (such as DMA, V3D, HVS, and ISP). Memory-mapped endpoints for the primary System bus are read synchronously, while the VideoCore VPU monitor is polled asynchronously in process context via a single bulk Raspberry Pi firmware mailbox read across all active bus watchers. Co-developed-by: Dom Cobley Signed-off-by: Dom Cobley Signed-off-by: Ian Rogers --- Note to maintainers: "depends on !RPI_AXIPERF" in drivers/perf/Kconfig prevents a conflict with the downstream Raspberry Pi debugfs driver (drivers/perf/raspberrypi_axi_monitor.c) and should be deleted when applying upstream. --- Documentation/admin-guide/perf/index.rst | 1 + .../admin-guide/perf/rpi-axi-pmu.rst | 133 + MAINTAINERS | 2 + drivers/perf/Kconfig | 14 + drivers/perf/Makefile | 1 + drivers/perf/rpi_axi_pmu.c | 2919 +++++++++++++++++ 6 files changed, 3070 insertions(+) create mode 100644 Documentation/admin-guide/perf/rpi-axi-pmu.rst create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/Documentation/admin-guide/perf/index.rst b/Documentation/admin-guide/perf/index.rst index aa12708ddb96..56414132600a 100644 --- a/Documentation/admin-guide/perf/index.rst +++ b/Documentation/admin-guide/perf/index.rst @@ -31,3 +31,4 @@ Performance monitor support ampere_cspmu mrvl-pem-pmu fujitsu_uncore_pmu + rpi-axi-pmu diff --git a/Documentation/admin-guide/perf/rpi-axi-pmu.rst b/Documentation/admin-guide/perf/rpi-axi-pmu.rst new file mode 100644 index 000000000000..d47ac3d6f2c2 --- /dev/null +++ b/Documentation/admin-guide/perf/rpi-axi-pmu.rst @@ -0,0 +1,133 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================================================== +Broadcom Raspberry Pi AXI Performance Monitoring Unit (PMU) +============================================================== + +Broadcom BCM2835 and BCM2711 SoCs used in Raspberry Pi boards include two AXI +bus-watcher performance monitor blocks: + +- **System monitor** (``monitor=0``): Observes the primary System AXI fabric + and is read directly via MMIO. +- **VPU monitor** (``monitor=1``): Observes the VideoCore VPU AXI fabric and + is polled asynchronously every 100 ms in process context through the + Raspberry Pi firmware mailbox (``RPI_FIRMWARE_GET_PERIPH_REG`` / + ``RPI_FIRMWARE_SET_PERIPH_REG``). If the running firmware does not grant + access to the VPU monitor registers, the ``vpu_*`` events are hidden from + sysfs at probe time. + +Each monitor provides three programmable hardware bus watchers. A single bus +watcher monitors one ``(bus, filter)`` pair and simultaneously tracks all +counter metrics for that bus and filter. Multiple perf events targeting the +same ``(monitor, bus, filter)`` share a single hardware bus watcher without +multiplexing; monitoring more than three distinct ``(bus, filter)`` pairs on +the same monitor multiplexes the watchers via perf. + +Event Format and Counters +========================= + +The PMU is registered under ``/sys/bus/event_source/devices/rpi_axi_pmu`` and +exposes the following ``format`` fields: + +- ``counter`` (``config:0-3``): Counter metric selector. +- ``bus`` (``config:4-8``): Monitored AXI bus index. +- ``monitor`` (``config:9-9``): ``0`` for System monitor, ``1`` for VPU + monitor. +- ``filter`` (``config:10-15``): Optional AXI master ID filter (``0`` disables + filtering and counts all masters on the selected bus). + +Each bus watcher provides the following counter metrics: + +======= ======== ====== ======================================================= +Counter Suffix Width Description +======= ======== ====== ======================================================= +0 atwait 28-bit Address transaction wait cycles +1 atrans 32-bit Address transactions (reads plus writes) +2 amax 28-bit Maximum address transaction latency (cycles) +3 wwait 28-bit Write data wait cycles +4 wtrans 32-bit Write data beats +5 wmax 28-bit Maximum write transaction latency (cycles) +6 rwait 28-bit Read data wait cycles +7 rtrans 32-bit Read data beats +8 rmax 28-bit Maximum read transaction latency (cycles) +9 rpend 8-bit Pending read transactions (instantaneous level) +======= ======== ====== ======================================================= + +Level Counters (``*max`` and ``rpend``) +--------------------------------------- + +Whereas ``*trans`` and ``*wait`` are cumulative counters, ``amax``, ``wmax``, +``rmax``, and ``rpend`` represent hardware levels/watermarks: + +- ``rpend`` is an instantaneous hardware register holding the number of reads + pending at the time of the register read. Across each ``perf stat`` read + interval, the driver tracks the peak ``rpend`` sampled and resets the + per-interval baseline on each read. +- ``amax``, ``wmax``, and ``rmax`` are sticky hardware peak-hold registers + that retain the maximum latency observed since the bus watcher was started + (as resetting the hardware bus watcher mid-run would clear concurrent + ``*trans`` and ``*wait`` counters sharing the watcher). + +Named Event Aliases and Master Filters +====================================== + +Sysfs event aliases are provided under ``events/`` in the form +``_`` for the System monitor and ``vpu__`` for the +VPU monitor. + +BCM2835 / BCM2837 +----------------- + +- **System buses (0..15):** ``dma_l2`` (0), ``trans`` (1), ``jpeg`` (2), + ``system_uc`` (3), ``dma_uc`` (4), ``system_l2`` (5), ``ccp2tx`` (6), + ``mphi_rx`` (7), ``mphi_tx`` (8), ``hvs`` (9), ``h264`` (10), ``isp`` (11), + ``v3d`` (12), ``peripheral`` (13), ``cpu_uc`` (14), ``cpu_l2`` (15). +- **VPU buses (0..14):** ``vpu_vpu1_d_l2`` (0), ``vpu_vpu0_d_l2`` (1), + ``vpu_vpu1_i_l2`` (2), ``vpu_vpu0_i_l2`` (3), ``vpu_system_l2`` (4), + ``vpu_dma_l2`` (5), ``vpu_vpu1_d_uc`` (6), ``vpu_vpu0_d_uc`` (7), + ``vpu_vpu1_i_uc`` (8), ``vpu_vpu0_i_uc`` (9), ``vpu_vpu_uc`` (10), + ``vpu_l2_out`` (11), ``vpu_dma_uc`` (12), ``vpu_l2_in`` (13), + ``vpu_sdram`` (14). +- **5-bit AXI master ID filters (1..31):** ``CORE0_V`` (1), ``ICACHE0`` (2), + ``DCACHE0`` (3), ``CORE1_V`` (4), ``ICACHE1`` (5), ``DCACHE1`` (6), + ``L2_MAIN`` (7), ``HOST_PORT`` (8), ``HOST_PORT2`` (9), ``HVS`` (10), + ``ISP`` (11), ``VIDEO_DCT`` (12), ``VIDEO_SD2AXI`` (13), ``CAM0`` (14), + ``CAM1`` (15), ``DMA0`` (16), ``DMA1`` (17), ``DMA2_VPU`` (18), + ``JPEG`` (19), ``VIDEO_CME`` (20), ``TRANSPOSER`` (21), ``VIDEO_FME`` (22), + ``CCP2TX`` (23), ``USB`` (24), ``V3D0`` (25), ``V3D1`` (26), ``V3D2`` (27), + ``AVE`` (28), ``DEBUG`` (29), ``CPU`` (30), ``M30`` (31). + +BCM2711 +------- + +- **System buses (0..11):** ``dma_l2`` (0), ``trans`` (1), ``jpeg`` (2), + ``vpu_uc`` (3), ``dma_uc`` (4), ``system_l2`` (5), ``hvs`` (6), + ``argon`` (7), ``h264`` (8), ``peripheral`` (9), ``arm_uc`` (10), + ``arm_l2`` (11). +- **VPU buses (0..13):** Same ``0..13`` layout as BCM2835 (without + ``vpu_sdram``). +- **5-bit AXI master ID filters (1..31):** Same as BCM2835 except ``ARGON`` + (8), ``PCIE`` (9), ``DMA2`` (18), ``GIGE`` (23), ``GISB_AXI`` (28), + ``ARM`` (30), and ``EMMCSTB`` (31). + +Examples +======== + +Count all metrics on the ``arm_uc`` bus during a memory copy workload (all +sharing a single hardware bus watcher):: + + perf stat -a \ + -e rpi_axi_pmu/arm_uc_atrans/ \ + -e rpi_axi_pmu/arm_uc_wtrans/ \ + -e rpi_axi_pmu/arm_uc_rtrans/ \ + -e rpi_axi_pmu/arm_uc_wmax/ \ + -e rpi_axi_pmu/arm_uc_rpend/ \ + -- dd if=/dev/zero of=/dev/null bs=1M count=512 + +Filter ``peripheral`` bus transactions by the ARM master ID (``filter=30`` on +BCM2835/BCM2711) at 1-second intervals:: + + perf stat -a -I 1000 \ + -e rpi_axi_pmu/peripheral_atrans/ \ + -e rpi_axi_pmu/peripheral_atrans,filter=30/ \ + -- sleep 5 diff --git a/MAINTAINERS b/MAINTAINERS index b8a3cb005c91..3a8da3ab52fc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5343,7 +5343,9 @@ M: Ian Rogers L: linux-perf-users@vger.kernel.org L: linux-rpi-kernel@lists.infradead.org (moderated for non-subscribers) S: Maintained +F: Documentation/admin-guide/perf/rpi-axi-pmu.rst F: Documentation/devicetree/bindings/perf/brcm,bcm2835-axiperf.yaml +F: drivers/perf/rpi_axi_pmu.c BROADCOM BCM2835 CAMERA DRIVERS M: Raspberry Pi Kernel Maintenance diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 245e7bb763b9..c1d7d64caba2 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -325,4 +325,18 @@ config NVIDIA_TEGRA410_C2C_PMU Enable perf support for counters in NVIDIA C2C interface of NVIDIA Tegra410 SoC. +config RPI_AXI_PMU + tristate "Raspberry Pi AXI Performance Monitor" + depends on ARCH_BCM2835 || COMPILE_TEST + depends on !RPI_AXIPERF + depends on RASPBERRYPI_FIRMWARE || RASPBERRYPI_FIRMWARE=n + help + Enable support for the Raspberry Pi AXI Performance Monitor. + This provides access to the System and VideoCore performance + counters on Broadcom BCM2835 and BCM2711 platforms. + + The System monitor is read directly via MMIO, while the VideoCore + VPU monitor is polled every 100ms via the Raspberry Pi firmware + mailbox. + endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index eb8a022dad9a..b9c98deb4379 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_MESON_DDR_PMU) += amlogic/ obj-$(CONFIG_CXL_PMU) += cxl_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_CMEM_LATENCY_PMU) += nvidia_t410_cmem_latency_pmu.o obj-$(CONFIG_NVIDIA_TEGRA410_C2C_PMU) += nvidia_t410_c2c_pmu.o +obj-$(CONFIG_RPI_AXI_PMU) += rpi_axi_pmu.o diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..f194b8633e43 --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,2919 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent monitors: + * 1. System Monitor (MON_SYSTEM = 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS, PCIe/RP1). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON_VPU = 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MMIO register updates + * (MON_SYSTEM) against SMP race conditions. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON_VPU) in process context, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON_VPU): + * Polled periodically in process context by vpu_work when active_vpu_events > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselines during + * event rotation / multiplexing. User read() syscalls return cached cumulative event counter + * instantly without blocking. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 (RPi 1-3, CM 1-3, Zero/W) + * @CHIP_BCM2711: BCM2711 (RPi 4, CM 4) + */ +enum rpi_axi_chip { + CHIP_BCM2835 = 0, + CHIP_BCM2711, +}; + +/** + * enum monitor - Hardware performance monitor blocks + * @MON_SYSTEM: System AXI monitor (MMIO) + * @MON_VPU: VideoCore VPU AXI monitor (Mailbox IPC) + * @MON_MAX: Total number of monitor blocks + */ +enum monitor { + MON_SYSTEM = 0, + MON_VPU, + MON_MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_SB_DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB_TRANS: Transposer engine bus (image format rotation & 2D matrix conversion) + * @BCM2835_SB_JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB_SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB_DMA_UC: DMA Uncached memory bus + * @BCM2835_SB_SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB_CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB_MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB_MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB_HVS: Hardware Video Scaler (HVS) multi-layer display composition engine bus + * @BCM2835_SB_H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB_ISP: Image Sensor Processor (ISP) camera processing pipeline bus + * @BCM2835_SB_V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB_PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, PWM, PCM) + * @BCM2835_SB_CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB_CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB_MAX: Total count of monitored system buses on BCM2835-BCM2711 + */ +enum bcm2835_system_bus { + BCM2835_SB_DMA_L2 = 0, + BCM2835_SB_TRANS = 1, + BCM2835_SB_JPEG = 2, + BCM2835_SB_SYSTEM_UC = 3, + BCM2835_SB_DMA_UC = 4, + BCM2835_SB_SYSTEM_L2 = 5, + BCM2835_SB_CCP2TX = 6, + BCM2835_SB_MPHI_RX = 7, + BCM2835_SB_MPHI_TX = 8, + BCM2835_SB_HVS = 9, + BCM2835_SB_H264 = 10, + BCM2835_SB_ISP = 11, + BCM2835_SB_V3D = 12, + BCM2835_SB_PERIPHERAL = 13, + BCM2835_SB_CPU_UC = 14, + BCM2835_SB_CPU_L2 = 15, + BCM2835_SB_MAX, +}; + +/** + * enum bcm2835_vpu_bus - AXI buses monitored by VPU Monitor on BCM2835 (RPi 1-3) + * @BCM2835_VB_VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @BCM2835_VB_VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @BCM2835_VB_VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @BCM2835_VB_VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @BCM2835_VB_SYSTEM_L2: VPU System L2 cache interconnect bus + * @BCM2835_VB_DMA_L2: VPU DMA L2 cache interconnect bus + * @BCM2835_VB_VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @BCM2835_VB_VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @BCM2835_VB_VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @BCM2835_VB_VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @BCM2835_VB_VPU_UC: VPU Uncached memory bus + * @BCM2835_VB_L2_OUT: VPU L2 cache outbound memory bus + * @BCM2835_VB_DMA_UC: VPU DMA Uncached memory bus + * @BCM2835_VB_L2_IN: VPU L2 cache inbound memory bus + * @BCM2835_VB_SDRAM: VPU SDRAM memory controller bus + * @BCM2835_VB_MAX: Total count of monitored VPU buses on BCM2835 + */ +enum bcm2835_vpu_bus { + BCM2835_VB_VPU1_D_L2 = 0, + BCM2835_VB_VPU0_D_L2 = 1, + BCM2835_VB_VPU1_I_L2 = 2, + BCM2835_VB_VPU0_I_L2 = 3, + BCM2835_VB_SYSTEM_L2 = 4, + BCM2835_VB_DMA_L2 = 5, + BCM2835_VB_VPU1_D_UC = 6, + BCM2835_VB_VPU0_D_UC = 7, + BCM2835_VB_VPU1_I_UC = 8, + BCM2835_VB_VPU0_I_UC = 9, + BCM2835_VB_VPU_UC = 10, + BCM2835_VB_L2_OUT = 11, + BCM2835_VB_DMA_UC = 12, + BCM2835_VB_L2_IN = 13, + BCM2835_VB_SDRAM = 14, + BCM2835_VB_MAX, +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watcher unit + * @CNT_ATWAIT: Total address phase wait / stall cycles + * @CNT_ATRANS: Total address phase transaction count + * @CNT_AMAX: Maximum address phase latency + * @CNT_WWAIT: Total write data phase wait / stall cycles + * @CNT_WTRANS: Total write data phase transaction count + * @CNT_WMAX: Maximum write data phase latency + * @CNT_RWAIT: Total read data phase wait / stall cycles + * @CNT_RTRANS: Total read data phase transaction count + * @CNT_RMAX: Maximum read data phase latency + * @CNT_RPEND: Total read pending cycles + * @CNT_RATRANS: Total read address phase transaction count + * @CNT_MAX: Total metric counters per watcher unit + */ +enum counter { + CNT_ATWAIT = 0, + CNT_ATRANS = 1, + CNT_AMAX = 2, + CNT_WWAIT = 3, + CNT_WTRANS = 4, + CNT_WMAX = 5, + CNT_RWAIT = 6, + CNT_RTRANS = 7, + CNT_RMAX = 8, + CNT_RPEND = 9, + CNT_RATRANS = 10, + CNT_MAX, +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 (RPi 1-4) + * @BCM2835_FLT_0: Disable master ID filtering (monitor all traffic on bus) + * @BCM2835_FLT_CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT_ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT_DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT_CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT_ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT_DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT_L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT_HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT_HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT_HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT_ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2835_FLT_VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accelerator master ID + * @BCM2835_FLT_VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT_CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT_CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT_DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT_DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT_DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT_JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT_VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2835_FLT_TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT_VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT_CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT_USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT_V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT_V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT_V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT_AVE: Audio-Video Engine master ID + * @BCM2835_FLT_DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT_CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT_M30: Reserved master ID slot 30 + * @BCM2835_FLT_MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT_0 = 0, + BCM2835_FLT_CORE0_V = 1, + BCM2835_FLT_ICACHE0 = 2, + BCM2835_FLT_DCACHE0 = 3, + BCM2835_FLT_CORE1_V = 4, + BCM2835_FLT_ICACHE1 = 5, + BCM2835_FLT_DCACHE1 = 6, + BCM2835_FLT_L2_MAIN = 7, + BCM2835_FLT_HOST_PORT = 8, + BCM2835_FLT_HOST_PORT2 = 9, + BCM2835_FLT_HVS = 10, + BCM2835_FLT_ISP = 11, + BCM2835_FLT_VIDEO_DCT = 12, + BCM2835_FLT_VIDEO_SD2AXI = 13, + BCM2835_FLT_CAM0 = 14, + BCM2835_FLT_CAM1 = 15, + BCM2835_FLT_DMA0 = 16, + BCM2835_FLT_DMA1 = 17, + BCM2835_FLT_DMA2_VPU = 18, + BCM2835_FLT_JPEG = 19, + BCM2835_FLT_VIDEO_CME = 20, + BCM2835_FLT_TRANSPOSER = 21, + BCM2835_FLT_VIDEO_FME = 22, + BCM2835_FLT_CCP2TX = 23, + BCM2835_FLT_USB = 24, + BCM2835_FLT_V3D0 = 25, + BCM2835_FLT_V3D1 = 26, + BCM2835_FLT_V3D2 = 27, + BCM2835_FLT_AVE = 28, + BCM2835_FLT_DEBUG = 29, + BCM2835_FLT_CPU = 30, + BCM2835_FLT_M30 = 31, + BCM2835_FLT_MAX, +}; + +/** + * enum bcm2711_system_bus - AXI buses monitored by System Monitor on BCM2711 (RPi 4) + * @BCM2711_SB_DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2711_SB_TRANS: Transposer engine bus + * @BCM2711_SB_JPEG: Hardware JPEG codec acceleration bus + * @BCM2711_SB_VPU_UC: VPU Uncached memory bus + * @BCM2711_SB_DMA_UC: DMA Uncached memory bus + * @BCM2711_SB_SYSTEM_L2: System main L2 cache bus + * @BCM2711_SB_HVS: Hardware Video Scaler (HVS) display engine bus + * @BCM2711_SB_ARGON: Argon video decoder bus + * @BCM2711_SB_H264: H.264 hardware video codec bus + * @BCM2711_SB_PERIPHERAL: System peripherals bus + * @BCM2711_SB_ARM_UC: ARM CPU Uncached memory bus + * @BCM2711_SB_ARM_L2: ARM CPU L2 cache bus + * @BCM2711_SB_MAX: Total count of monitored system buses on BCM2711 + */ +enum bcm2711_system_bus { + BCM2711_SB_DMA_L2 = 0, + BCM2711_SB_TRANS = 1, + BCM2711_SB_JPEG = 2, + BCM2711_SB_VPU_UC = 3, + BCM2711_SB_DMA_UC = 4, + BCM2711_SB_SYSTEM_L2 = 5, + BCM2711_SB_HVS = 6, + BCM2711_SB_ARGON = 7, + BCM2711_SB_H264 = 8, + BCM2711_SB_PERIPHERAL = 9, + BCM2711_SB_ARM_UC = 10, + BCM2711_SB_ARM_L2 = 11, + BCM2711_SB_MAX, +}; + +/** + * enum bcm2711_vpu_bus - AXI buses monitored by VPU Monitor on BCM2711 (RPi 4) + * @BCM2711_VB_VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @BCM2711_VB_VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @BCM2711_VB_VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @BCM2711_VB_VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @BCM2711_VB_SYSTEM_L2: VPU System L2 cache interconnect bus + * @BCM2711_VB_DMA_L2: VPU DMA L2 cache interconnect bus + * @BCM2711_VB_VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @BCM2711_VB_VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @BCM2711_VB_VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @BCM2711_VB_VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @BCM2711_VB_VPU_UC: VPU Uncached memory bus + * @BCM2711_VB_L2_OUT: VPU L2 cache outbound memory bus + * @BCM2711_VB_DMA_UC: VPU DMA Uncached memory bus + * @BCM2711_VB_L2_IN: VPU L2 cache inbound memory bus + * @BCM2711_VB_MAX: Total count of monitored VPU buses on BCM2711 + */ +enum bcm2711_vpu_bus { + BCM2711_VB_VPU1_D_L2 = 0, + BCM2711_VB_VPU0_D_L2 = 1, + BCM2711_VB_VPU1_I_L2 = 2, + BCM2711_VB_VPU0_I_L2 = 3, + BCM2711_VB_SYSTEM_L2 = 4, + BCM2711_VB_DMA_L2 = 5, + BCM2711_VB_VPU1_D_UC = 6, + BCM2711_VB_VPU0_D_UC = 7, + BCM2711_VB_VPU1_I_UC = 8, + BCM2711_VB_VPU0_I_UC = 9, + BCM2711_VB_VPU_UC = 10, + BCM2711_VB_L2_OUT = 11, + BCM2711_VB_DMA_UC = 12, + BCM2711_VB_L2_IN = 13, + BCM2711_VB_MAX, +}; + +/** + * enum bcm2711_filter - AXI master ID filter options for BCM2711 (RPi 4) + * @BCM2711_FLT_AIO: Audio/Video I/O master ID (or 0 to disable filtering) + * @BCM2711_FLT_CORE0_V: VideoCore Core 0 master ID + * @BCM2711_FLT_ICACHE0: VideoCore Core 0 Instruction Cache master ID + * @BCM2711_FLT_DCACHE0: VideoCore Core 0 Data Cache master ID + * @BCM2711_FLT_CORE1_V: VideoCore Core 1 master ID + * @BCM2711_FLT_ICACHE1: VideoCore Core 1 Instruction Cache master ID + * @BCM2711_FLT_DCACHE1: VideoCore Core 1 Data Cache master ID + * @BCM2711_FLT_L2_MAIN: Main L2 cache controller master ID + * @BCM2711_FLT_ARGON: Argon video decoder master ID + * @BCM2711_FLT_PCIE: PCIe controller master ID + * @BCM2711_FLT_HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2711_FLT_ISP: Image Sensor Processor (ISP) camera pipeline master ID + * @BCM2711_FLT_VIDEO_DCT: Discrete Cosine Transform (DCT) accelerator master ID + * @BCM2711_FLT_VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2711_FLT_CAM0: Camera Unicam 0 receiver master ID + * @BCM2711_FLT_CAM1: Camera Unicam 1 receiver master ID + * @BCM2711_FLT_DMA0: System DMA Channel 0 master ID + * @BCM2711_FLT_DMA1: System DMA Channel 1 master ID + * @BCM2711_FLT_DMA2: VPU DMA engine 2 master ID + * @BCM2711_FLT_JPEG: JPEG decoder hardware master ID + * @BCM2711_FLT_VIDEO_CME: Motion Estimation hardware accelerator master ID + * @BCM2711_FLT_TRANSPOSER: Image Transposer engine master ID + * @BCM2711_FLT_VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2711_FLT_GIGE: Gigabit Ethernet controller master ID + * @BCM2711_FLT_USB: USB controller master ID + * @BCM2711_FLT_V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2711_FLT_V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2711_FLT_V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2711_FLT_GISB_AXI: GISB to AXI bridge master ID + * @BCM2711_FLT_DEBUG: Debug unit master ID + * @BCM2711_FLT_ARM: ARM CPU cluster master ID + * @BCM2711_FLT_EMMCSTB: EMMC STB controller master ID + * @BCM2711_FLT_MAX: Maximum filter ID count for BCM2711 + */ +enum bcm2711_filter { + BCM2711_FLT_AIO = 0, + BCM2711_FLT_CORE0_V = 1, + BCM2711_FLT_ICACHE0 = 2, + BCM2711_FLT_DCACHE0 = 3, + BCM2711_FLT_CORE1_V = 4, + BCM2711_FLT_ICACHE1 = 5, + BCM2711_FLT_DCACHE1 = 6, + BCM2711_FLT_L2_MAIN = 7, + BCM2711_FLT_ARGON = 8, + BCM2711_FLT_PCIE = 9, + BCM2711_FLT_HVS = 10, + BCM2711_FLT_ISP = 11, + BCM2711_FLT_VIDEO_DCT = 12, + BCM2711_FLT_VIDEO_SD2AXI = 13, + BCM2711_FLT_CAM0 = 14, + BCM2711_FLT_CAM1 = 15, + BCM2711_FLT_DMA0 = 16, + BCM2711_FLT_DMA1 = 17, + BCM2711_FLT_DMA2 = 18, + BCM2711_FLT_JPEG = 19, + BCM2711_FLT_VIDEO_CME = 20, + BCM2711_FLT_TRANSPOSER = 21, + BCM2711_FLT_VIDEO_FME = 22, + BCM2711_FLT_GIGE = 23, + BCM2711_FLT_USB = 24, + BCM2711_FLT_V3D0 = 25, + BCM2711_FLT_V3D1 = 26, + BCM2711_FLT_V3D2 = 27, + BCM2711_FLT_GISB_AXI = 28, + BCM2711_FLT_DEBUG = 29, + BCM2711_FLT_ARM = 30, + BCM2711_FLT_EMMCSTB = 31, + BCM2711_FLT_MAX, +}; + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_STRIDE 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +/* + * Event counter registers are contiguous and logically relative to their + * respective Watcher Control register. These offsets are dynamically added + * to the computed base (BWn_CTRL) to derive the physical address. + */ +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 +#define BW_RPEND_OFFSET 0x28 +#define BW_RATRANS_OFFSET 0x2c + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(4, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +/* + * RPI_AXI_PMU_TIMER_INTERVAL determines the background polling frequency + * for VideoCore VPU Mailbox IPC counters. + * + * A balance is required: + * - IPC Overhead: Polling overly fast (e.g., 10ms) generates excessive CPU + * wakeups and VideoCore IPC interrupts on older CPUs (Pi 1/2). + * - Accuracy: Polling overly slow (e.g., 2000ms) causes short time-multiplexed + * profiling sessions (under the interval) to mathematically strand residual + * counts since the mailbox cannot be queried synchronously inside pmu->read(). + * + * 100ms (10 Hz) provides reasonably accurate profiling without heavy overhead. + */ +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(100) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; +/* --- PMU API & CONFIG DECODING ---------------------------------- */ +#define PMU_NAME "rpi_axi_pmu" + +/* + * perf_event_attr config format: + * [10-15] : Filter ID + * [9] : Monitor ID (0 = System, 1 = VPU) + * [4-8] : Bus index + * [0-3] : Counter enum + */ +#define RPI_AXI_CFG_FILTER_SHIFT 10 +#define RPI_AXI_CFG_FILTER_MASK 0x3F +#define RPI_AXI_CFG_MONITOR_SHIFT 9 +#define RPI_AXI_CFG_MONITOR_MASK 0x1 +#define RPI_AXI_CFG_BUS_SHIFT 4 +#define RPI_AXI_CFG_BUS_MASK 0x1F +#define RPI_AXI_CFG_COUNTER_MASK 0xF + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-15). + */ +static int config_to_filter(__u64 config) +{ + return (config >> RPI_AXI_CFG_FILTER_SHIFT) & RPI_AXI_CFG_FILTER_MASK; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 = System, 1 = VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> RPI_AXI_CFG_MONITOR_SHIFT) & RPI_AXI_CFG_MONITOR_MASK; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> RPI_AXI_CFG_BUS_SHIFT) & RPI_AXI_CFG_BUS_MASK; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & RPI_AXI_CFG_COUNTER_MASK; +} + +struct rpi_axi_pmu; +/** + * config_is_valid() - Validates whether event config bitfields match SoC capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of the 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @enabled: Hardware enabled status for each watcher + * @monitor_running: Flag indicating if the hardware monitor loop is globally active + * @vpu_disable_pending: Array tracking asynchronous hardware disable requests for VPU + * @config_gen: Bumped whenever a watcher is allocated or freed + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + bool monitor_running; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; + bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR]; + unsigned int config_gen[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored = 0; + hw_events->monitor_running = false; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] = -1; + hw_events->filter[i] = 0; + hw_events->refcount[i] = 0; + hw_events->enabled[i] = false; + hw_events->vpu_disable_pending[i] = false; + hw_events->config_gen[i] = 0; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus watcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events *hw_events, + const struct perf_event *event) +{ + int bus = config_to_bus(event->attr.config); + int filter = config_to_filter(event->attr.config); + + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == bus && hw_events->filter[i] == filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored == NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + for (int i = 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] == -1) { + hw_events->monitored_bus[i] = bus; + hw_events->filter[i] = filter; + hw_events->refcount[i] = 1; + hw_events->vpu_disable_pending[i] = false; + hw_events->config_gen[i]++; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 66 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835) + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BCM2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO counter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in process context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow polling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + int cpu; + struct hlist_node cpuhp_node; + raw_spinlock_t lock; + struct mutex vpu_mutex; + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON_MAX]; +}; + +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon = config_to_monitor(config); + int bus = config_to_bus(config); + int filter = config_to_filter(config); + int counter = config_to_counter(config); + + if (config >> 16 != 0) + return false; + + if (mon >= MON_MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_address) + return false; + + if (filter >= 64) + return false; + + if (counter == CNT_RATRANS && + (pmu->chip == CHIP_BCM2835 || pmu->chip == CHIP_BCM2711)) + return false; + + switch (pmu->chip) { + case CHIP_BCM2835: + if (mon == MON_SYSTEM && bus >= BCM2835_SB_MAX) + return false; + if (mon == MON_VPU && bus >= BCM2835_VB_MAX) + return false; + if (filter >= BCM2835_FLT_MAX) + return false; + break; + case CHIP_BCM2711: + if (mon == MON_SYSTEM && bus >= BCM2711_SB_MAX) + return false; + if (mon == MON_VPU && bus >= BCM2711_VB_MAX) + return false; + if (filter >= BCM2711_FLT_MAX) + return false; + break; + } + + return counter < CNT_MAX; +} + +PMU_EVENT_ATTR_STRING(dma_l2_atwait, bcm2835_dma_l2_atwait, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atrans, bcm2835_dma_l2_atrans, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_amax, bcm2835_dma_l2_amax, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wwait, bcm2835_dma_l2_wwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, bcm2835_dma_l2_wtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_wmax, bcm2835_dma_l2_wmax, "monitor=0,bus=0,counter=5"); +PMU_EVENT_ATTR_STRING(dma_l2_rwait, bcm2835_dma_l2_rwait, "monitor=0,bus=0,counter=6"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, bcm2835_dma_l2_rtrans, "monitor=0,bus=0,counter=7"); +PMU_EVENT_ATTR_STRING(dma_l2_rmax, bcm2835_dma_l2_rmax, "monitor=0,bus=0,counter=8"); +PMU_EVENT_ATTR_STRING(dma_l2_rpend, bcm2835_dma_l2_rpend, "monitor=0,bus=0,counter=9"); +PMU_EVENT_ATTR_STRING(trans_atwait, bcm2835_trans_atwait, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atrans, bcm2835_trans_atrans, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_amax, bcm2835_trans_amax, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wwait, bcm2835_trans_wwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_wtrans, bcm2835_trans_wtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_wmax, bcm2835_trans_wmax, "monitor=0,bus=1,counter=5"); +PMU_EVENT_ATTR_STRING(trans_rwait, bcm2835_trans_rwait, "monitor=0,bus=1,counter=6"); +PMU_EVENT_ATTR_STRING(trans_rtrans, bcm2835_trans_rtrans, "monitor=0,bus=1,counter=7"); +PMU_EVENT_ATTR_STRING(trans_rmax, bcm2835_trans_rmax, "monitor=0,bus=1,counter=8"); +PMU_EVENT_ATTR_STRING(trans_rpend, bcm2835_trans_rpend, "monitor=0,bus=1,counter=9"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, bcm2835_jpeg_atwait, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atrans, bcm2835_jpeg_atrans, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_amax, bcm2835_jpeg_amax, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wwait, bcm2835_jpeg_wwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, bcm2835_jpeg_wtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_wmax, bcm2835_jpeg_wmax, "monitor=0,bus=2,counter=5"); +PMU_EVENT_ATTR_STRING(jpeg_rwait, bcm2835_jpeg_rwait, "monitor=0,bus=2,counter=6"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, bcm2835_jpeg_rtrans, "monitor=0,bus=2,counter=7"); +PMU_EVENT_ATTR_STRING(jpeg_rmax, bcm2835_jpeg_rmax, "monitor=0,bus=2,counter=8"); +PMU_EVENT_ATTR_STRING(jpeg_rpend, bcm2835_jpeg_rpend, "monitor=0,bus=2,counter=9"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, bcm2835_system_uc_atwait, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(system_uc_atrans, bcm2835_system_uc_atrans, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(system_uc_amax, bcm2835_system_uc_amax, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(system_uc_wwait, bcm2835_system_uc_wwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, bcm2835_system_uc_wtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(system_uc_wmax, bcm2835_system_uc_wmax, "monitor=0,bus=3,counter=5"); +PMU_EVENT_ATTR_STRING(system_uc_rwait, bcm2835_system_uc_rwait, "monitor=0,bus=3,counter=6"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, bcm2835_system_uc_rtrans, "monitor=0,bus=3,counter=7"); +PMU_EVENT_ATTR_STRING(system_uc_rmax, bcm2835_system_uc_rmax, "monitor=0,bus=3,counter=8"); +PMU_EVENT_ATTR_STRING(system_uc_rpend, bcm2835_system_uc_rpend, "monitor=0,bus=3,counter=9"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, bcm2835_dma_uc_atwait, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atrans, bcm2835_dma_uc_atrans, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_amax, bcm2835_dma_uc_amax, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wwait, bcm2835_dma_uc_wwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, bcm2835_dma_uc_wtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_wmax, bcm2835_dma_uc_wmax, "monitor=0,bus=4,counter=5"); +PMU_EVENT_ATTR_STRING(dma_uc_rwait, bcm2835_dma_uc_rwait, "monitor=0,bus=4,counter=6"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, bcm2835_dma_uc_rtrans, "monitor=0,bus=4,counter=7"); +PMU_EVENT_ATTR_STRING(dma_uc_rmax, bcm2835_dma_uc_rmax, "monitor=0,bus=4,counter=8"); +PMU_EVENT_ATTR_STRING(dma_uc_rpend, bcm2835_dma_uc_rpend, "monitor=0,bus=4,counter=9"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, bcm2835_system_l2_atwait, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atrans, bcm2835_system_l2_atrans, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_amax, bcm2835_system_l2_amax, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wwait, bcm2835_system_l2_wwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, bcm2835_system_l2_wtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_wmax, bcm2835_system_l2_wmax, "monitor=0,bus=5,counter=5"); +PMU_EVENT_ATTR_STRING(system_l2_rwait, bcm2835_system_l2_rwait, "monitor=0,bus=5,counter=6"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, bcm2835_system_l2_rtrans, "monitor=0,bus=5,counter=7"); +PMU_EVENT_ATTR_STRING(system_l2_rmax, bcm2835_system_l2_rmax, "monitor=0,bus=5,counter=8"); +PMU_EVENT_ATTR_STRING(system_l2_rpend, bcm2835_system_l2_rpend, "monitor=0,bus=5,counter=9"); +PMU_EVENT_ATTR_STRING(ccp2tx_atwait, bcm2835_ccp2tx_atwait, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, bcm2835_ccp2tx_atrans, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(ccp2tx_amax, bcm2835_ccp2tx_amax, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(ccp2tx_wwait, bcm2835_ccp2tx_wwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(ccp2tx_wtrans, bcm2835_ccp2tx_wtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(ccp2tx_wmax, bcm2835_ccp2tx_wmax, "monitor=0,bus=6,counter=5"); +PMU_EVENT_ATTR_STRING(ccp2tx_rwait, bcm2835_ccp2tx_rwait, "monitor=0,bus=6,counter=6"); +PMU_EVENT_ATTR_STRING(ccp2tx_rtrans, bcm2835_ccp2tx_rtrans, "monitor=0,bus=6,counter=7"); +PMU_EVENT_ATTR_STRING(ccp2tx_rmax, bcm2835_ccp2tx_rmax, "monitor=0,bus=6,counter=8"); +PMU_EVENT_ATTR_STRING(ccp2tx_rpend, bcm2835_ccp2tx_rpend, "monitor=0,bus=6,counter=9"); +PMU_EVENT_ATTR_STRING(mphi_rx_atwait, bcm2835_mphi_rx_atwait, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, bcm2835_mphi_rx_atrans, "monitor=0,bus=7,counter=1"); +PMU_EVENT_ATTR_STRING(mphi_rx_amax, bcm2835_mphi_rx_amax, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(mphi_rx_wwait, bcm2835_mphi_rx_wwait, "monitor=0,bus=7,counter=3"); +PMU_EVENT_ATTR_STRING(mphi_rx_wtrans, bcm2835_mphi_rx_wtrans, "monitor=0,bus=7,counter=4"); +PMU_EVENT_ATTR_STRING(mphi_rx_wmax, bcm2835_mphi_rx_wmax, "monitor=0,bus=7,counter=5"); +PMU_EVENT_ATTR_STRING(mphi_rx_rwait, bcm2835_mphi_rx_rwait, "monitor=0,bus=7,counter=6"); +PMU_EVENT_ATTR_STRING(mphi_rx_rtrans, bcm2835_mphi_rx_rtrans, "monitor=0,bus=7,counter=7"); +PMU_EVENT_ATTR_STRING(mphi_rx_rmax, bcm2835_mphi_rx_rmax, "monitor=0,bus=7,counter=8"); +PMU_EVENT_ATTR_STRING(mphi_rx_rpend, bcm2835_mphi_rx_rpend, "monitor=0,bus=7,counter=9"); +PMU_EVENT_ATTR_STRING(mphi_tx_atwait, bcm2835_mphi_tx_atwait, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, bcm2835_mphi_tx_atrans, "monitor=0,bus=8,counter=1"); +PMU_EVENT_ATTR_STRING(mphi_tx_amax, bcm2835_mphi_tx_amax, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(mphi_tx_wwait, bcm2835_mphi_tx_wwait, "monitor=0,bus=8,counter=3"); +PMU_EVENT_ATTR_STRING(mphi_tx_wtrans, bcm2835_mphi_tx_wtrans, "monitor=0,bus=8,counter=4"); +PMU_EVENT_ATTR_STRING(mphi_tx_wmax, bcm2835_mphi_tx_wmax, "monitor=0,bus=8,counter=5"); +PMU_EVENT_ATTR_STRING(mphi_tx_rwait, bcm2835_mphi_tx_rwait, "monitor=0,bus=8,counter=6"); +PMU_EVENT_ATTR_STRING(mphi_tx_rtrans, bcm2835_mphi_tx_rtrans, "monitor=0,bus=8,counter=7"); +PMU_EVENT_ATTR_STRING(mphi_tx_rmax, bcm2835_mphi_tx_rmax, "monitor=0,bus=8,counter=8"); +PMU_EVENT_ATTR_STRING(mphi_tx_rpend, bcm2835_mphi_tx_rpend, "monitor=0,bus=8,counter=9"); +PMU_EVENT_ATTR_STRING(hvs_atwait, bcm2835_hvs_atwait, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atrans, bcm2835_hvs_atrans, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_amax, bcm2835_hvs_amax, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wwait, bcm2835_hvs_wwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, bcm2835_hvs_wtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_wmax, bcm2835_hvs_wmax, "monitor=0,bus=9,counter=5"); +PMU_EVENT_ATTR_STRING(hvs_rwait, bcm2835_hvs_rwait, "monitor=0,bus=9,counter=6"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, bcm2835_hvs_rtrans, "monitor=0,bus=9,counter=7"); +PMU_EVENT_ATTR_STRING(hvs_rmax, bcm2835_hvs_rmax, "monitor=0,bus=9,counter=8"); +PMU_EVENT_ATTR_STRING(hvs_rpend, bcm2835_hvs_rpend, "monitor=0,bus=9,counter=9"); +PMU_EVENT_ATTR_STRING(h264_atwait, bcm2835_h264_atwait, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atrans, bcm2835_h264_atrans, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(h264_amax, bcm2835_h264_amax, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wwait, bcm2835_h264_wwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(h264_wtrans, bcm2835_h264_wtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(h264_wmax, bcm2835_h264_wmax, "monitor=0,bus=10,counter=5"); +PMU_EVENT_ATTR_STRING(h264_rwait, bcm2835_h264_rwait, "monitor=0,bus=10,counter=6"); +PMU_EVENT_ATTR_STRING(h264_rtrans, bcm2835_h264_rtrans, "monitor=0,bus=10,counter=7"); +PMU_EVENT_ATTR_STRING(h264_rmax, bcm2835_h264_rmax, "monitor=0,bus=10,counter=8"); +PMU_EVENT_ATTR_STRING(h264_rpend, bcm2835_h264_rpend, "monitor=0,bus=10,counter=9"); +PMU_EVENT_ATTR_STRING(isp_atwait, bcm2835_isp_atwait, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(isp_atrans, bcm2835_isp_atrans, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(isp_amax, bcm2835_isp_amax, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(isp_wwait, bcm2835_isp_wwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(isp_wtrans, bcm2835_isp_wtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(isp_wmax, bcm2835_isp_wmax, "monitor=0,bus=11,counter=5"); +PMU_EVENT_ATTR_STRING(isp_rwait, bcm2835_isp_rwait, "monitor=0,bus=11,counter=6"); +PMU_EVENT_ATTR_STRING(isp_rtrans, bcm2835_isp_rtrans, "monitor=0,bus=11,counter=7"); +PMU_EVENT_ATTR_STRING(isp_rmax, bcm2835_isp_rmax, "monitor=0,bus=11,counter=8"); +PMU_EVENT_ATTR_STRING(isp_rpend, bcm2835_isp_rpend, "monitor=0,bus=11,counter=9"); +PMU_EVENT_ATTR_STRING(v3d_atwait, bcm2835_v3d_atwait, "monitor=0,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(v3d_atrans, bcm2835_v3d_atrans, "monitor=0,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(v3d_amax, bcm2835_v3d_amax, "monitor=0,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(v3d_wwait, bcm2835_v3d_wwait, "monitor=0,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, bcm2835_v3d_wtrans, "monitor=0,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(v3d_wmax, bcm2835_v3d_wmax, "monitor=0,bus=12,counter=5"); +PMU_EVENT_ATTR_STRING(v3d_rwait, bcm2835_v3d_rwait, "monitor=0,bus=12,counter=6"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, bcm2835_v3d_rtrans, "monitor=0,bus=12,counter=7"); +PMU_EVENT_ATTR_STRING(v3d_rmax, bcm2835_v3d_rmax, "monitor=0,bus=12,counter=8"); +PMU_EVENT_ATTR_STRING(v3d_rpend, bcm2835_v3d_rpend, "monitor=0,bus=12,counter=9"); +PMU_EVENT_ATTR_STRING(peripheral_atwait, bcm2835_peripheral_atwait, "monitor=0,bus=13,counter=0"); +PMU_EVENT_ATTR_STRING(peripheral_atrans, bcm2835_peripheral_atrans, "monitor=0,bus=13,counter=1"); +PMU_EVENT_ATTR_STRING(peripheral_amax, bcm2835_peripheral_amax, "monitor=0,bus=13,counter=2"); +PMU_EVENT_ATTR_STRING(peripheral_wwait, bcm2835_peripheral_wwait, "monitor=0,bus=13,counter=3"); +PMU_EVENT_ATTR_STRING(peripheral_wtrans, bcm2835_peripheral_wtrans, "monitor=0,bus=13,counter=4"); +PMU_EVENT_ATTR_STRING(peripheral_wmax, bcm2835_peripheral_wmax, "monitor=0,bus=13,counter=5"); +PMU_EVENT_ATTR_STRING(peripheral_rwait, bcm2835_peripheral_rwait, "monitor=0,bus=13,counter=6"); +PMU_EVENT_ATTR_STRING(peripheral_rtrans, bcm2835_peripheral_rtrans, "monitor=0,bus=13,counter=7"); +PMU_EVENT_ATTR_STRING(peripheral_rmax, bcm2835_peripheral_rmax, "monitor=0,bus=13,counter=8"); +PMU_EVENT_ATTR_STRING(peripheral_rpend, bcm2835_peripheral_rpend, "monitor=0,bus=13,counter=9"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, bcm2835_cpu_uc_atwait, "monitor=0,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, bcm2835_cpu_uc_atrans, "monitor=0,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_uc_amax, bcm2835_cpu_uc_amax, "monitor=0,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wwait, bcm2835_cpu_uc_wwait, "monitor=0,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, bcm2835_cpu_uc_wtrans, "monitor=0,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_uc_wmax, bcm2835_cpu_uc_wmax, "monitor=0,bus=14,counter=5"); +PMU_EVENT_ATTR_STRING(cpu_uc_rwait, bcm2835_cpu_uc_rwait, "monitor=0,bus=14,counter=6"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, bcm2835_cpu_uc_rtrans, "monitor=0,bus=14,counter=7"); +PMU_EVENT_ATTR_STRING(cpu_uc_rmax, bcm2835_cpu_uc_rmax, "monitor=0,bus=14,counter=8"); +PMU_EVENT_ATTR_STRING(cpu_uc_rpend, bcm2835_cpu_uc_rpend, "monitor=0,bus=14,counter=9"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, bcm2835_cpu_l2_atwait, "monitor=0,bus=15,counter=0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, bcm2835_cpu_l2_atrans, "monitor=0,bus=15,counter=1"); +PMU_EVENT_ATTR_STRING(cpu_l2_amax, bcm2835_cpu_l2_amax, "monitor=0,bus=15,counter=2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wwait, bcm2835_cpu_l2_wwait, "monitor=0,bus=15,counter=3"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, bcm2835_cpu_l2_wtrans, "monitor=0,bus=15,counter=4"); +PMU_EVENT_ATTR_STRING(cpu_l2_wmax, bcm2835_cpu_l2_wmax, "monitor=0,bus=15,counter=5"); +PMU_EVENT_ATTR_STRING(cpu_l2_rwait, bcm2835_cpu_l2_rwait, "monitor=0,bus=15,counter=6"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, bcm2835_cpu_l2_rtrans, "monitor=0,bus=15,counter=7"); +PMU_EVENT_ATTR_STRING(cpu_l2_rmax, bcm2835_cpu_l2_rmax, "monitor=0,bus=15,counter=8"); +PMU_EVENT_ATTR_STRING(cpu_l2_rpend, bcm2835_cpu_l2_rpend, "monitor=0,bus=15,counter=9"); + +PMU_FORMAT_ATTR(monitor, "config:9-9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); +PMU_FORMAT_ATTR(filter, "config:10-15"); + +static struct attribute *rpi_axi_pmu_format_attrs[] = { + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + &format_attr_filter.attr, + NULL +}; + +static const struct attribute_group rpi_axi_pmu_format_group = { + .name = "format", + .attrs = rpi_axi_pmu_format_attrs, +}; + +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct pmu *pmu = dev_get_drvdata(dev); + struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu); + + return sysfs_emit(buf, "%*pbl\n", cpumask_pr_args(cpumask_of(rpi_pmu->cpu))); +} +static DEVICE_ATTR_RO(cpumask); +static struct attribute *rpi_axi_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group = { + .attrs = rpi_axi_pmu_cpumask_attrs, +}; + +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int n) +{ + struct device *dev = kobj_to_dev(kobj); + struct pmu *pmu = dev_get_drvdata(dev); + struct rpi_axi_pmu *rpi_pmu = pmu_to_rpi_axi_pmu(pmu); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr.attr); + if (!rpi_pmu->monitor[MON_VPU].use_mailbox_interface && + pmu_attr->event_str && + !strncmp(pmu_attr->event_str, "monitor=1,", 10)) + return 0; + + return attr->mode; +} + +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_atwait, bcm2835_vpu_vpu1_d_l2_atwait, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_atrans, bcm2835_vpu_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_amax, bcm2835_vpu_vpu1_d_l2_amax, "monitor=1,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_wwait, bcm2835_vpu_vpu1_d_l2_wwait, "monitor=1,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_wtrans, bcm2835_vpu_vpu1_d_l2_wtrans, "monitor=1,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_wmax, bcm2835_vpu_vpu1_d_l2_wmax, "monitor=1,bus=0,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rwait, bcm2835_vpu_vpu1_d_l2_rwait, "monitor=1,bus=0,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rtrans, bcm2835_vpu_vpu1_d_l2_rtrans, "monitor=1,bus=0,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rmax, bcm2835_vpu_vpu1_d_l2_rmax, "monitor=1,bus=0,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rpend, bcm2835_vpu_vpu1_d_l2_rpend, "monitor=1,bus=0,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_atwait, bcm2835_vpu_vpu0_d_l2_atwait, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_atrans, bcm2835_vpu_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_amax, bcm2835_vpu_vpu0_d_l2_amax, "monitor=1,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_wwait, bcm2835_vpu_vpu0_d_l2_wwait, "monitor=1,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_wtrans, bcm2835_vpu_vpu0_d_l2_wtrans, "monitor=1,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_wmax, bcm2835_vpu_vpu0_d_l2_wmax, "monitor=1,bus=1,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rwait, bcm2835_vpu_vpu0_d_l2_rwait, "monitor=1,bus=1,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rtrans, bcm2835_vpu_vpu0_d_l2_rtrans, "monitor=1,bus=1,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rmax, bcm2835_vpu_vpu0_d_l2_rmax, "monitor=1,bus=1,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rpend, bcm2835_vpu_vpu0_d_l2_rpend, "monitor=1,bus=1,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_atwait, bcm2835_vpu_vpu1_i_l2_atwait, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_atrans, bcm2835_vpu_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_amax, bcm2835_vpu_vpu1_i_l2_amax, "monitor=1,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_wwait, bcm2835_vpu_vpu1_i_l2_wwait, "monitor=1,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_wtrans, bcm2835_vpu_vpu1_i_l2_wtrans, "monitor=1,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_wmax, bcm2835_vpu_vpu1_i_l2_wmax, "monitor=1,bus=2,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rwait, bcm2835_vpu_vpu1_i_l2_rwait, "monitor=1,bus=2,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rtrans, bcm2835_vpu_vpu1_i_l2_rtrans, "monitor=1,bus=2,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rmax, bcm2835_vpu_vpu1_i_l2_rmax, "monitor=1,bus=2,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rpend, bcm2835_vpu_vpu1_i_l2_rpend, "monitor=1,bus=2,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_atwait, bcm2835_vpu_vpu0_i_l2_atwait, "monitor=1,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_atrans, bcm2835_vpu_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_amax, bcm2835_vpu_vpu0_i_l2_amax, "monitor=1,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_wwait, bcm2835_vpu_vpu0_i_l2_wwait, "monitor=1,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_wtrans, bcm2835_vpu_vpu0_i_l2_wtrans, "monitor=1,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_wmax, bcm2835_vpu_vpu0_i_l2_wmax, "monitor=1,bus=3,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rwait, bcm2835_vpu_vpu0_i_l2_rwait, "monitor=1,bus=3,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rtrans, bcm2835_vpu_vpu0_i_l2_rtrans, "monitor=1,bus=3,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rmax, bcm2835_vpu_vpu0_i_l2_rmax, "monitor=1,bus=3,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rpend, bcm2835_vpu_vpu0_i_l2_rpend, "monitor=1,bus=3,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_atwait, bcm2835_vpu_system_l2_atwait, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, bcm2835_vpu_system_l2_atrans, "monitor=1,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_amax, bcm2835_vpu_system_l2_amax, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wwait, bcm2835_vpu_system_l2_wwait, "monitor=1,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, bcm2835_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wmax, bcm2835_vpu_system_l2_wmax, "monitor=1,bus=4,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rwait, bcm2835_vpu_system_l2_rwait, "monitor=1,bus=4,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, bcm2835_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rmax, bcm2835_vpu_system_l2_rmax, "monitor=1,bus=4,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rpend, bcm2835_vpu_system_l2_rpend, "monitor=1,bus=4,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atwait, bcm2835_vpu_dma_l2_atwait, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, bcm2835_vpu_dma_l2_atrans, "monitor=1,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_amax, bcm2835_vpu_dma_l2_amax, "monitor=1,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_wwait, bcm2835_vpu_dma_l2_wwait, "monitor=1,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_wtrans, bcm2835_vpu_dma_l2_wtrans, "monitor=1,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_wmax, bcm2835_vpu_dma_l2_wmax, "monitor=1,bus=5,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rwait, bcm2835_vpu_dma_l2_rwait, "monitor=1,bus=5,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rtrans, bcm2835_vpu_dma_l2_rtrans, "monitor=1,bus=5,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rmax, bcm2835_vpu_dma_l2_rmax, "monitor=1,bus=5,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rpend, bcm2835_vpu_dma_l2_rpend, "monitor=1,bus=5,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_atwait, bcm2835_vpu_vpu1_d_uc_atwait, "monitor=1,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_atrans, bcm2835_vpu_vpu1_d_uc_atrans, "monitor=1,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_amax, bcm2835_vpu_vpu1_d_uc_amax, "monitor=1,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_wwait, bcm2835_vpu_vpu1_d_uc_wwait, "monitor=1,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_wtrans, bcm2835_vpu_vpu1_d_uc_wtrans, "monitor=1,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_wmax, bcm2835_vpu_vpu1_d_uc_wmax, "monitor=1,bus=6,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rwait, bcm2835_vpu_vpu1_d_uc_rwait, "monitor=1,bus=6,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rtrans, bcm2835_vpu_vpu1_d_uc_rtrans, "monitor=1,bus=6,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rmax, bcm2835_vpu_vpu1_d_uc_rmax, "monitor=1,bus=6,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rpend, bcm2835_vpu_vpu1_d_uc_rpend, "monitor=1,bus=6,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_atwait, bcm2835_vpu_vpu0_d_uc_atwait, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_atrans, bcm2835_vpu_vpu0_d_uc_atrans, "monitor=1,bus=7,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_amax, bcm2835_vpu_vpu0_d_uc_amax, "monitor=1,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_wwait, bcm2835_vpu_vpu0_d_uc_wwait, "monitor=1,bus=7,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_wtrans, bcm2835_vpu_vpu0_d_uc_wtrans, "monitor=1,bus=7,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_wmax, bcm2835_vpu_vpu0_d_uc_wmax, "monitor=1,bus=7,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rwait, bcm2835_vpu_vpu0_d_uc_rwait, "monitor=1,bus=7,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rtrans, bcm2835_vpu_vpu0_d_uc_rtrans, "monitor=1,bus=7,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rmax, bcm2835_vpu_vpu0_d_uc_rmax, "monitor=1,bus=7,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rpend, bcm2835_vpu_vpu0_d_uc_rpend, "monitor=1,bus=7,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_atwait, bcm2835_vpu_vpu1_i_uc_atwait, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_atrans, bcm2835_vpu_vpu1_i_uc_atrans, "monitor=1,bus=8,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_amax, bcm2835_vpu_vpu1_i_uc_amax, "monitor=1,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_wwait, bcm2835_vpu_vpu1_i_uc_wwait, "monitor=1,bus=8,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_wtrans, bcm2835_vpu_vpu1_i_uc_wtrans, "monitor=1,bus=8,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_wmax, bcm2835_vpu_vpu1_i_uc_wmax, "monitor=1,bus=8,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rwait, bcm2835_vpu_vpu1_i_uc_rwait, "monitor=1,bus=8,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rtrans, bcm2835_vpu_vpu1_i_uc_rtrans, "monitor=1,bus=8,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rmax, bcm2835_vpu_vpu1_i_uc_rmax, "monitor=1,bus=8,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rpend, bcm2835_vpu_vpu1_i_uc_rpend, "monitor=1,bus=8,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_atwait, bcm2835_vpu_vpu0_i_uc_atwait, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_atrans, bcm2835_vpu_vpu0_i_uc_atrans, "monitor=1,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_amax, bcm2835_vpu_vpu0_i_uc_amax, "monitor=1,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_wwait, bcm2835_vpu_vpu0_i_uc_wwait, "monitor=1,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_wtrans, bcm2835_vpu_vpu0_i_uc_wtrans, "monitor=1,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_wmax, bcm2835_vpu_vpu0_i_uc_wmax, "monitor=1,bus=9,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rwait, bcm2835_vpu_vpu0_i_uc_rwait, "monitor=1,bus=9,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rtrans, bcm2835_vpu_vpu0_i_uc_rtrans, "monitor=1,bus=9,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rmax, bcm2835_vpu_vpu0_i_uc_rmax, "monitor=1,bus=9,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rpend, bcm2835_vpu_vpu0_i_uc_rpend, "monitor=1,bus=9,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_atwait, bcm2835_vpu_vpu_uc_atwait, "monitor=1,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_atrans, bcm2835_vpu_vpu_uc_atrans, "monitor=1,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_amax, bcm2835_vpu_vpu_uc_amax, "monitor=1,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_wwait, bcm2835_vpu_vpu_uc_wwait, "monitor=1,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_wtrans, bcm2835_vpu_vpu_uc_wtrans, "monitor=1,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_wmax, bcm2835_vpu_vpu_uc_wmax, "monitor=1,bus=10,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rwait, bcm2835_vpu_vpu_uc_rwait, "monitor=1,bus=10,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rtrans, bcm2835_vpu_vpu_uc_rtrans, "monitor=1,bus=10,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rmax, bcm2835_vpu_vpu_uc_rmax, "monitor=1,bus=10,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rpend, bcm2835_vpu_vpu_uc_rpend, "monitor=1,bus=10,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_atwait, bcm2835_vpu_l2_out_atwait, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, bcm2835_vpu_l2_out_atrans, "monitor=1,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_amax, bcm2835_vpu_l2_out_amax, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_wwait, bcm2835_vpu_l2_out_wwait, "monitor=1,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_wtrans, bcm2835_vpu_l2_out_wtrans, "monitor=1,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_wmax, bcm2835_vpu_l2_out_wmax, "monitor=1,bus=11,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rwait, bcm2835_vpu_l2_out_rwait, "monitor=1,bus=11,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rtrans, bcm2835_vpu_l2_out_rtrans, "monitor=1,bus=11,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rmax, bcm2835_vpu_l2_out_rmax, "monitor=1,bus=11,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rpend, bcm2835_vpu_l2_out_rpend, "monitor=1,bus=11,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atwait, bcm2835_vpu_dma_uc_atwait, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, bcm2835_vpu_dma_uc_atrans, "monitor=1,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_amax, bcm2835_vpu_dma_uc_amax, "monitor=1,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_wwait, bcm2835_vpu_dma_uc_wwait, "monitor=1,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_wtrans, bcm2835_vpu_dma_uc_wtrans, "monitor=1,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_wmax, bcm2835_vpu_dma_uc_wmax, "monitor=1,bus=12,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rwait, bcm2835_vpu_dma_uc_rwait, "monitor=1,bus=12,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rtrans, bcm2835_vpu_dma_uc_rtrans, "monitor=1,bus=12,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rmax, bcm2835_vpu_dma_uc_rmax, "monitor=1,bus=12,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rpend, bcm2835_vpu_dma_uc_rpend, "monitor=1,bus=12,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_atwait, bcm2835_vpu_l2_in_atwait, "monitor=1,bus=13,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, bcm2835_vpu_l2_in_atrans, "monitor=1,bus=13,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_amax, bcm2835_vpu_l2_in_amax, "monitor=1,bus=13,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_wwait, bcm2835_vpu_l2_in_wwait, "monitor=1,bus=13,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_wtrans, bcm2835_vpu_l2_in_wtrans, "monitor=1,bus=13,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_wmax, bcm2835_vpu_l2_in_wmax, "monitor=1,bus=13,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rwait, bcm2835_vpu_l2_in_rwait, "monitor=1,bus=13,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rtrans, bcm2835_vpu_l2_in_rtrans, "monitor=1,bus=13,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rmax, bcm2835_vpu_l2_in_rmax, "monitor=1,bus=13,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rpend, bcm2835_vpu_l2_in_rpend, "monitor=1,bus=13,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, bcm2835_vpu_sdram_atwait, "monitor=1,bus=14,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, bcm2835_vpu_sdram_atrans, "monitor=1,bus=14,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_amax, bcm2835_vpu_sdram_amax, "monitor=1,bus=14,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wwait, bcm2835_vpu_sdram_wwait, "monitor=1,bus=14,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, bcm2835_vpu_sdram_wtrans, "monitor=1,bus=14,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wmax, bcm2835_vpu_sdram_wmax, "monitor=1,bus=14,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rwait, bcm2835_vpu_sdram_rwait, "monitor=1,bus=14,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, bcm2835_vpu_sdram_rtrans, "monitor=1,bus=14,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rmax, bcm2835_vpu_sdram_rmax, "monitor=1,bus=14,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rpend, bcm2835_vpu_sdram_rpend, "monitor=1,bus=14,counter=9"); + +static struct attribute *bcm2835_events[] = { + &bcm2835_dma_l2_atwait.attr.attr, + &bcm2835_dma_l2_atrans.attr.attr, + &bcm2835_dma_l2_amax.attr.attr, + &bcm2835_dma_l2_wwait.attr.attr, + &bcm2835_dma_l2_wtrans.attr.attr, + &bcm2835_dma_l2_wmax.attr.attr, + &bcm2835_dma_l2_rwait.attr.attr, + &bcm2835_dma_l2_rtrans.attr.attr, + &bcm2835_dma_l2_rmax.attr.attr, + &bcm2835_dma_l2_rpend.attr.attr, + &bcm2835_trans_atwait.attr.attr, + &bcm2835_trans_atrans.attr.attr, + &bcm2835_trans_amax.attr.attr, + &bcm2835_trans_wwait.attr.attr, + &bcm2835_trans_wtrans.attr.attr, + &bcm2835_trans_wmax.attr.attr, + &bcm2835_trans_rwait.attr.attr, + &bcm2835_trans_rtrans.attr.attr, + &bcm2835_trans_rmax.attr.attr, + &bcm2835_trans_rpend.attr.attr, + &bcm2835_jpeg_atwait.attr.attr, + &bcm2835_jpeg_atrans.attr.attr, + &bcm2835_jpeg_amax.attr.attr, + &bcm2835_jpeg_wwait.attr.attr, + &bcm2835_jpeg_wtrans.attr.attr, + &bcm2835_jpeg_wmax.attr.attr, + &bcm2835_jpeg_rwait.attr.attr, + &bcm2835_jpeg_rtrans.attr.attr, + &bcm2835_jpeg_rmax.attr.attr, + &bcm2835_jpeg_rpend.attr.attr, + &bcm2835_system_uc_atwait.attr.attr, + &bcm2835_system_uc_atrans.attr.attr, + &bcm2835_system_uc_amax.attr.attr, + &bcm2835_system_uc_wwait.attr.attr, + &bcm2835_system_uc_wtrans.attr.attr, + &bcm2835_system_uc_wmax.attr.attr, + &bcm2835_system_uc_rwait.attr.attr, + &bcm2835_system_uc_rtrans.attr.attr, + &bcm2835_system_uc_rmax.attr.attr, + &bcm2835_system_uc_rpend.attr.attr, + &bcm2835_dma_uc_atwait.attr.attr, + &bcm2835_dma_uc_atrans.attr.attr, + &bcm2835_dma_uc_amax.attr.attr, + &bcm2835_dma_uc_wwait.attr.attr, + &bcm2835_dma_uc_wtrans.attr.attr, + &bcm2835_dma_uc_wmax.attr.attr, + &bcm2835_dma_uc_rwait.attr.attr, + &bcm2835_dma_uc_rtrans.attr.attr, + &bcm2835_dma_uc_rmax.attr.attr, + &bcm2835_dma_uc_rpend.attr.attr, + &bcm2835_system_l2_atwait.attr.attr, + &bcm2835_system_l2_atrans.attr.attr, + &bcm2835_system_l2_amax.attr.attr, + &bcm2835_system_l2_wwait.attr.attr, + &bcm2835_system_l2_wtrans.attr.attr, + &bcm2835_system_l2_wmax.attr.attr, + &bcm2835_system_l2_rwait.attr.attr, + &bcm2835_system_l2_rtrans.attr.attr, + &bcm2835_system_l2_rmax.attr.attr, + &bcm2835_system_l2_rpend.attr.attr, + &bcm2835_ccp2tx_atwait.attr.attr, + &bcm2835_ccp2tx_atrans.attr.attr, + &bcm2835_ccp2tx_amax.attr.attr, + &bcm2835_ccp2tx_wwait.attr.attr, + &bcm2835_ccp2tx_wtrans.attr.attr, + &bcm2835_ccp2tx_wmax.attr.attr, + &bcm2835_ccp2tx_rwait.attr.attr, + &bcm2835_ccp2tx_rtrans.attr.attr, + &bcm2835_ccp2tx_rmax.attr.attr, + &bcm2835_ccp2tx_rpend.attr.attr, + &bcm2835_mphi_rx_atwait.attr.attr, + &bcm2835_mphi_rx_atrans.attr.attr, + &bcm2835_mphi_rx_amax.attr.attr, + &bcm2835_mphi_rx_wwait.attr.attr, + &bcm2835_mphi_rx_wtrans.attr.attr, + &bcm2835_mphi_rx_wmax.attr.attr, + &bcm2835_mphi_rx_rwait.attr.attr, + &bcm2835_mphi_rx_rtrans.attr.attr, + &bcm2835_mphi_rx_rmax.attr.attr, + &bcm2835_mphi_rx_rpend.attr.attr, + &bcm2835_mphi_tx_atwait.attr.attr, + &bcm2835_mphi_tx_atrans.attr.attr, + &bcm2835_mphi_tx_amax.attr.attr, + &bcm2835_mphi_tx_wwait.attr.attr, + &bcm2835_mphi_tx_wtrans.attr.attr, + &bcm2835_mphi_tx_wmax.attr.attr, + &bcm2835_mphi_tx_rwait.attr.attr, + &bcm2835_mphi_tx_rtrans.attr.attr, + &bcm2835_mphi_tx_rmax.attr.attr, + &bcm2835_mphi_tx_rpend.attr.attr, + &bcm2835_hvs_atwait.attr.attr, + &bcm2835_hvs_atrans.attr.attr, + &bcm2835_hvs_amax.attr.attr, + &bcm2835_hvs_wwait.attr.attr, + &bcm2835_hvs_wtrans.attr.attr, + &bcm2835_hvs_wmax.attr.attr, + &bcm2835_hvs_rwait.attr.attr, + &bcm2835_hvs_rtrans.attr.attr, + &bcm2835_hvs_rmax.attr.attr, + &bcm2835_hvs_rpend.attr.attr, + &bcm2835_h264_atwait.attr.attr, + &bcm2835_h264_atrans.attr.attr, + &bcm2835_h264_amax.attr.attr, + &bcm2835_h264_wwait.attr.attr, + &bcm2835_h264_wtrans.attr.attr, + &bcm2835_h264_wmax.attr.attr, + &bcm2835_h264_rwait.attr.attr, + &bcm2835_h264_rtrans.attr.attr, + &bcm2835_h264_rmax.attr.attr, + &bcm2835_h264_rpend.attr.attr, + &bcm2835_isp_atwait.attr.attr, + &bcm2835_isp_atrans.attr.attr, + &bcm2835_isp_amax.attr.attr, + &bcm2835_isp_wwait.attr.attr, + &bcm2835_isp_wtrans.attr.attr, + &bcm2835_isp_wmax.attr.attr, + &bcm2835_isp_rwait.attr.attr, + &bcm2835_isp_rtrans.attr.attr, + &bcm2835_isp_rmax.attr.attr, + &bcm2835_isp_rpend.attr.attr, + &bcm2835_v3d_atwait.attr.attr, + &bcm2835_v3d_atrans.attr.attr, + &bcm2835_v3d_amax.attr.attr, + &bcm2835_v3d_wwait.attr.attr, + &bcm2835_v3d_wtrans.attr.attr, + &bcm2835_v3d_wmax.attr.attr, + &bcm2835_v3d_rwait.attr.attr, + &bcm2835_v3d_rtrans.attr.attr, + &bcm2835_v3d_rmax.attr.attr, + &bcm2835_v3d_rpend.attr.attr, + &bcm2835_peripheral_atwait.attr.attr, + &bcm2835_peripheral_atrans.attr.attr, + &bcm2835_peripheral_amax.attr.attr, + &bcm2835_peripheral_wwait.attr.attr, + &bcm2835_peripheral_wtrans.attr.attr, + &bcm2835_peripheral_wmax.attr.attr, + &bcm2835_peripheral_rwait.attr.attr, + &bcm2835_peripheral_rtrans.attr.attr, + &bcm2835_peripheral_rmax.attr.attr, + &bcm2835_peripheral_rpend.attr.attr, + &bcm2835_cpu_uc_atwait.attr.attr, + &bcm2835_cpu_uc_atrans.attr.attr, + &bcm2835_cpu_uc_amax.attr.attr, + &bcm2835_cpu_uc_wwait.attr.attr, + &bcm2835_cpu_uc_wtrans.attr.attr, + &bcm2835_cpu_uc_wmax.attr.attr, + &bcm2835_cpu_uc_rwait.attr.attr, + &bcm2835_cpu_uc_rtrans.attr.attr, + &bcm2835_cpu_uc_rmax.attr.attr, + &bcm2835_cpu_uc_rpend.attr.attr, + &bcm2835_cpu_l2_atwait.attr.attr, + &bcm2835_cpu_l2_atrans.attr.attr, + &bcm2835_cpu_l2_amax.attr.attr, + &bcm2835_cpu_l2_wwait.attr.attr, + &bcm2835_cpu_l2_wtrans.attr.attr, + &bcm2835_cpu_l2_wmax.attr.attr, + &bcm2835_cpu_l2_rwait.attr.attr, + &bcm2835_cpu_l2_rtrans.attr.attr, + &bcm2835_cpu_l2_rmax.attr.attr, + &bcm2835_cpu_l2_rpend.attr.attr, + &bcm2835_vpu_vpu1_d_l2_atwait.attr.attr, + &bcm2835_vpu_vpu1_d_l2_atrans.attr.attr, + &bcm2835_vpu_vpu1_d_l2_amax.attr.attr, + &bcm2835_vpu_vpu1_d_l2_wwait.attr.attr, + &bcm2835_vpu_vpu1_d_l2_wtrans.attr.attr, + &bcm2835_vpu_vpu1_d_l2_wmax.attr.attr, + &bcm2835_vpu_vpu1_d_l2_rwait.attr.attr, + &bcm2835_vpu_vpu1_d_l2_rtrans.attr.attr, + &bcm2835_vpu_vpu1_d_l2_rmax.attr.attr, + &bcm2835_vpu_vpu1_d_l2_rpend.attr.attr, + &bcm2835_vpu_vpu0_d_l2_atwait.attr.attr, + &bcm2835_vpu_vpu0_d_l2_atrans.attr.attr, + &bcm2835_vpu_vpu0_d_l2_amax.attr.attr, + &bcm2835_vpu_vpu0_d_l2_wwait.attr.attr, + &bcm2835_vpu_vpu0_d_l2_wtrans.attr.attr, + &bcm2835_vpu_vpu0_d_l2_wmax.attr.attr, + &bcm2835_vpu_vpu0_d_l2_rwait.attr.attr, + &bcm2835_vpu_vpu0_d_l2_rtrans.attr.attr, + &bcm2835_vpu_vpu0_d_l2_rmax.attr.attr, + &bcm2835_vpu_vpu0_d_l2_rpend.attr.attr, + &bcm2835_vpu_vpu1_i_l2_atwait.attr.attr, + &bcm2835_vpu_vpu1_i_l2_atrans.attr.attr, + &bcm2835_vpu_vpu1_i_l2_amax.attr.attr, + &bcm2835_vpu_vpu1_i_l2_wwait.attr.attr, + &bcm2835_vpu_vpu1_i_l2_wtrans.attr.attr, + &bcm2835_vpu_vpu1_i_l2_wmax.attr.attr, + &bcm2835_vpu_vpu1_i_l2_rwait.attr.attr, + &bcm2835_vpu_vpu1_i_l2_rtrans.attr.attr, + &bcm2835_vpu_vpu1_i_l2_rmax.attr.attr, + &bcm2835_vpu_vpu1_i_l2_rpend.attr.attr, + &bcm2835_vpu_vpu0_i_l2_atwait.attr.attr, + &bcm2835_vpu_vpu0_i_l2_atrans.attr.attr, + &bcm2835_vpu_vpu0_i_l2_amax.attr.attr, + &bcm2835_vpu_vpu0_i_l2_wwait.attr.attr, + &bcm2835_vpu_vpu0_i_l2_wtrans.attr.attr, + &bcm2835_vpu_vpu0_i_l2_wmax.attr.attr, + &bcm2835_vpu_vpu0_i_l2_rwait.attr.attr, + &bcm2835_vpu_vpu0_i_l2_rtrans.attr.attr, + &bcm2835_vpu_vpu0_i_l2_rmax.attr.attr, + &bcm2835_vpu_vpu0_i_l2_rpend.attr.attr, + &bcm2835_vpu_system_l2_atwait.attr.attr, + &bcm2835_vpu_system_l2_atrans.attr.attr, + &bcm2835_vpu_system_l2_amax.attr.attr, + &bcm2835_vpu_system_l2_wwait.attr.attr, + &bcm2835_vpu_system_l2_wtrans.attr.attr, + &bcm2835_vpu_system_l2_wmax.attr.attr, + &bcm2835_vpu_system_l2_rwait.attr.attr, + &bcm2835_vpu_system_l2_rtrans.attr.attr, + &bcm2835_vpu_system_l2_rmax.attr.attr, + &bcm2835_vpu_system_l2_rpend.attr.attr, + &bcm2835_vpu_dma_l2_atwait.attr.attr, + &bcm2835_vpu_dma_l2_atrans.attr.attr, + &bcm2835_vpu_dma_l2_amax.attr.attr, + &bcm2835_vpu_dma_l2_wwait.attr.attr, + &bcm2835_vpu_dma_l2_wtrans.attr.attr, + &bcm2835_vpu_dma_l2_wmax.attr.attr, + &bcm2835_vpu_dma_l2_rwait.attr.attr, + &bcm2835_vpu_dma_l2_rtrans.attr.attr, + &bcm2835_vpu_dma_l2_rmax.attr.attr, + &bcm2835_vpu_dma_l2_rpend.attr.attr, + &bcm2835_vpu_vpu1_d_uc_atwait.attr.attr, + &bcm2835_vpu_vpu1_d_uc_atrans.attr.attr, + &bcm2835_vpu_vpu1_d_uc_amax.attr.attr, + &bcm2835_vpu_vpu1_d_uc_wwait.attr.attr, + &bcm2835_vpu_vpu1_d_uc_wtrans.attr.attr, + &bcm2835_vpu_vpu1_d_uc_wmax.attr.attr, + &bcm2835_vpu_vpu1_d_uc_rwait.attr.attr, + &bcm2835_vpu_vpu1_d_uc_rtrans.attr.attr, + &bcm2835_vpu_vpu1_d_uc_rmax.attr.attr, + &bcm2835_vpu_vpu1_d_uc_rpend.attr.attr, + &bcm2835_vpu_vpu0_d_uc_atwait.attr.attr, + &bcm2835_vpu_vpu0_d_uc_atrans.attr.attr, + &bcm2835_vpu_vpu0_d_uc_amax.attr.attr, + &bcm2835_vpu_vpu0_d_uc_wwait.attr.attr, + &bcm2835_vpu_vpu0_d_uc_wtrans.attr.attr, + &bcm2835_vpu_vpu0_d_uc_wmax.attr.attr, + &bcm2835_vpu_vpu0_d_uc_rwait.attr.attr, + &bcm2835_vpu_vpu0_d_uc_rtrans.attr.attr, + &bcm2835_vpu_vpu0_d_uc_rmax.attr.attr, + &bcm2835_vpu_vpu0_d_uc_rpend.attr.attr, + &bcm2835_vpu_vpu1_i_uc_atwait.attr.attr, + &bcm2835_vpu_vpu1_i_uc_atrans.attr.attr, + &bcm2835_vpu_vpu1_i_uc_amax.attr.attr, + &bcm2835_vpu_vpu1_i_uc_wwait.attr.attr, + &bcm2835_vpu_vpu1_i_uc_wtrans.attr.attr, + &bcm2835_vpu_vpu1_i_uc_wmax.attr.attr, + &bcm2835_vpu_vpu1_i_uc_rwait.attr.attr, + &bcm2835_vpu_vpu1_i_uc_rtrans.attr.attr, + &bcm2835_vpu_vpu1_i_uc_rmax.attr.attr, + &bcm2835_vpu_vpu1_i_uc_rpend.attr.attr, + &bcm2835_vpu_vpu0_i_uc_atwait.attr.attr, + &bcm2835_vpu_vpu0_i_uc_atrans.attr.attr, + &bcm2835_vpu_vpu0_i_uc_amax.attr.attr, + &bcm2835_vpu_vpu0_i_uc_wwait.attr.attr, + &bcm2835_vpu_vpu0_i_uc_wtrans.attr.attr, + &bcm2835_vpu_vpu0_i_uc_wmax.attr.attr, + &bcm2835_vpu_vpu0_i_uc_rwait.attr.attr, + &bcm2835_vpu_vpu0_i_uc_rtrans.attr.attr, + &bcm2835_vpu_vpu0_i_uc_rmax.attr.attr, + &bcm2835_vpu_vpu0_i_uc_rpend.attr.attr, + &bcm2835_vpu_vpu_uc_atwait.attr.attr, + &bcm2835_vpu_vpu_uc_atrans.attr.attr, + &bcm2835_vpu_vpu_uc_amax.attr.attr, + &bcm2835_vpu_vpu_uc_wwait.attr.attr, + &bcm2835_vpu_vpu_uc_wtrans.attr.attr, + &bcm2835_vpu_vpu_uc_wmax.attr.attr, + &bcm2835_vpu_vpu_uc_rwait.attr.attr, + &bcm2835_vpu_vpu_uc_rtrans.attr.attr, + &bcm2835_vpu_vpu_uc_rmax.attr.attr, + &bcm2835_vpu_vpu_uc_rpend.attr.attr, + &bcm2835_vpu_l2_out_atwait.attr.attr, + &bcm2835_vpu_l2_out_atrans.attr.attr, + &bcm2835_vpu_l2_out_amax.attr.attr, + &bcm2835_vpu_l2_out_wwait.attr.attr, + &bcm2835_vpu_l2_out_wtrans.attr.attr, + &bcm2835_vpu_l2_out_wmax.attr.attr, + &bcm2835_vpu_l2_out_rwait.attr.attr, + &bcm2835_vpu_l2_out_rtrans.attr.attr, + &bcm2835_vpu_l2_out_rmax.attr.attr, + &bcm2835_vpu_l2_out_rpend.attr.attr, + &bcm2835_vpu_dma_uc_atwait.attr.attr, + &bcm2835_vpu_dma_uc_atrans.attr.attr, + &bcm2835_vpu_dma_uc_amax.attr.attr, + &bcm2835_vpu_dma_uc_wwait.attr.attr, + &bcm2835_vpu_dma_uc_wtrans.attr.attr, + &bcm2835_vpu_dma_uc_wmax.attr.attr, + &bcm2835_vpu_dma_uc_rwait.attr.attr, + &bcm2835_vpu_dma_uc_rtrans.attr.attr, + &bcm2835_vpu_dma_uc_rmax.attr.attr, + &bcm2835_vpu_dma_uc_rpend.attr.attr, + &bcm2835_vpu_l2_in_atwait.attr.attr, + &bcm2835_vpu_l2_in_atrans.attr.attr, + &bcm2835_vpu_l2_in_amax.attr.attr, + &bcm2835_vpu_l2_in_wwait.attr.attr, + &bcm2835_vpu_l2_in_wtrans.attr.attr, + &bcm2835_vpu_l2_in_wmax.attr.attr, + &bcm2835_vpu_l2_in_rwait.attr.attr, + &bcm2835_vpu_l2_in_rtrans.attr.attr, + &bcm2835_vpu_l2_in_rmax.attr.attr, + &bcm2835_vpu_l2_in_rpend.attr.attr, + &bcm2835_vpu_sdram_atwait.attr.attr, + &bcm2835_vpu_sdram_atrans.attr.attr, + &bcm2835_vpu_sdram_amax.attr.attr, + &bcm2835_vpu_sdram_wwait.attr.attr, + &bcm2835_vpu_sdram_wtrans.attr.attr, + &bcm2835_vpu_sdram_wmax.attr.attr, + &bcm2835_vpu_sdram_rwait.attr.attr, + &bcm2835_vpu_sdram_rtrans.attr.attr, + &bcm2835_vpu_sdram_rmax.attr.attr, + &bcm2835_vpu_sdram_rpend.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_bcm2835_events_group = { + .name = "events", + .attrs = bcm2835_events, + .is_visible = rpi_axi_pmu_events_is_visible, +}; + +PMU_EVENT_ATTR_STRING(dma_l2_atwait, bcm2711_dma_l2_atwait, "monitor=0,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(dma_l2_atrans, bcm2711_dma_l2_atrans, "monitor=0,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(dma_l2_amax, bcm2711_dma_l2_amax, "monitor=0,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(dma_l2_wwait, bcm2711_dma_l2_wwait, "monitor=0,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, bcm2711_dma_l2_wtrans, "monitor=0,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(dma_l2_wmax, bcm2711_dma_l2_wmax, "monitor=0,bus=0,counter=5"); +PMU_EVENT_ATTR_STRING(dma_l2_rwait, bcm2711_dma_l2_rwait, "monitor=0,bus=0,counter=6"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, bcm2711_dma_l2_rtrans, "monitor=0,bus=0,counter=7"); +PMU_EVENT_ATTR_STRING(dma_l2_rmax, bcm2711_dma_l2_rmax, "monitor=0,bus=0,counter=8"); +PMU_EVENT_ATTR_STRING(dma_l2_rpend, bcm2711_dma_l2_rpend, "monitor=0,bus=0,counter=9"); +PMU_EVENT_ATTR_STRING(trans_atwait, bcm2711_trans_atwait, "monitor=0,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(trans_atrans, bcm2711_trans_atrans, "monitor=0,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(trans_amax, bcm2711_trans_amax, "monitor=0,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(trans_wwait, bcm2711_trans_wwait, "monitor=0,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(trans_wtrans, bcm2711_trans_wtrans, "monitor=0,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(trans_wmax, bcm2711_trans_wmax, "monitor=0,bus=1,counter=5"); +PMU_EVENT_ATTR_STRING(trans_rwait, bcm2711_trans_rwait, "monitor=0,bus=1,counter=6"); +PMU_EVENT_ATTR_STRING(trans_rtrans, bcm2711_trans_rtrans, "monitor=0,bus=1,counter=7"); +PMU_EVENT_ATTR_STRING(trans_rmax, bcm2711_trans_rmax, "monitor=0,bus=1,counter=8"); +PMU_EVENT_ATTR_STRING(trans_rpend, bcm2711_trans_rpend, "monitor=0,bus=1,counter=9"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, bcm2711_jpeg_atwait, "monitor=0,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(jpeg_atrans, bcm2711_jpeg_atrans, "monitor=0,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(jpeg_amax, bcm2711_jpeg_amax, "monitor=0,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(jpeg_wwait, bcm2711_jpeg_wwait, "monitor=0,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, bcm2711_jpeg_wtrans, "monitor=0,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(jpeg_wmax, bcm2711_jpeg_wmax, "monitor=0,bus=2,counter=5"); +PMU_EVENT_ATTR_STRING(jpeg_rwait, bcm2711_jpeg_rwait, "monitor=0,bus=2,counter=6"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, bcm2711_jpeg_rtrans, "monitor=0,bus=2,counter=7"); +PMU_EVENT_ATTR_STRING(jpeg_rmax, bcm2711_jpeg_rmax, "monitor=0,bus=2,counter=8"); +PMU_EVENT_ATTR_STRING(jpeg_rpend, bcm2711_jpeg_rpend, "monitor=0,bus=2,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_uc_atwait, bcm2711_vpu_uc_atwait, "monitor=0,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_uc_atrans, bcm2711_vpu_uc_atrans, "monitor=0,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_uc_amax, bcm2711_vpu_uc_amax, "monitor=0,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_uc_wwait, bcm2711_vpu_uc_wwait, "monitor=0,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_uc_wtrans, bcm2711_vpu_uc_wtrans, "monitor=0,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_uc_wmax, bcm2711_vpu_uc_wmax, "monitor=0,bus=3,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_uc_rwait, bcm2711_vpu_uc_rwait, "monitor=0,bus=3,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_uc_rtrans, bcm2711_vpu_uc_rtrans, "monitor=0,bus=3,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_uc_rmax, bcm2711_vpu_uc_rmax, "monitor=0,bus=3,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_uc_rpend, bcm2711_vpu_uc_rpend, "monitor=0,bus=3,counter=9"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, bcm2711_dma_uc_atwait, "monitor=0,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(dma_uc_atrans, bcm2711_dma_uc_atrans, "monitor=0,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(dma_uc_amax, bcm2711_dma_uc_amax, "monitor=0,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(dma_uc_wwait, bcm2711_dma_uc_wwait, "monitor=0,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, bcm2711_dma_uc_wtrans, "monitor=0,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(dma_uc_wmax, bcm2711_dma_uc_wmax, "monitor=0,bus=4,counter=5"); +PMU_EVENT_ATTR_STRING(dma_uc_rwait, bcm2711_dma_uc_rwait, "monitor=0,bus=4,counter=6"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, bcm2711_dma_uc_rtrans, "monitor=0,bus=4,counter=7"); +PMU_EVENT_ATTR_STRING(dma_uc_rmax, bcm2711_dma_uc_rmax, "monitor=0,bus=4,counter=8"); +PMU_EVENT_ATTR_STRING(dma_uc_rpend, bcm2711_dma_uc_rpend, "monitor=0,bus=4,counter=9"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, bcm2711_system_l2_atwait, "monitor=0,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(system_l2_atrans, bcm2711_system_l2_atrans, "monitor=0,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(system_l2_amax, bcm2711_system_l2_amax, "monitor=0,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(system_l2_wwait, bcm2711_system_l2_wwait, "monitor=0,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, bcm2711_system_l2_wtrans, "monitor=0,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(system_l2_wmax, bcm2711_system_l2_wmax, "monitor=0,bus=5,counter=5"); +PMU_EVENT_ATTR_STRING(system_l2_rwait, bcm2711_system_l2_rwait, "monitor=0,bus=5,counter=6"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, bcm2711_system_l2_rtrans, "monitor=0,bus=5,counter=7"); +PMU_EVENT_ATTR_STRING(system_l2_rmax, bcm2711_system_l2_rmax, "monitor=0,bus=5,counter=8"); +PMU_EVENT_ATTR_STRING(system_l2_rpend, bcm2711_system_l2_rpend, "monitor=0,bus=5,counter=9"); +PMU_EVENT_ATTR_STRING(hvs_atwait, bcm2711_hvs_atwait, "monitor=0,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(hvs_atrans, bcm2711_hvs_atrans, "monitor=0,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(hvs_amax, bcm2711_hvs_amax, "monitor=0,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(hvs_wwait, bcm2711_hvs_wwait, "monitor=0,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, bcm2711_hvs_wtrans, "monitor=0,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(hvs_wmax, bcm2711_hvs_wmax, "monitor=0,bus=6,counter=5"); +PMU_EVENT_ATTR_STRING(hvs_rwait, bcm2711_hvs_rwait, "monitor=0,bus=6,counter=6"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, bcm2711_hvs_rtrans, "monitor=0,bus=6,counter=7"); +PMU_EVENT_ATTR_STRING(hvs_rmax, bcm2711_hvs_rmax, "monitor=0,bus=6,counter=8"); +PMU_EVENT_ATTR_STRING(hvs_rpend, bcm2711_hvs_rpend, "monitor=0,bus=6,counter=9"); +PMU_EVENT_ATTR_STRING(argon_atwait, bcm2711_argon_atwait, "monitor=0,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(argon_atrans, bcm2711_argon_atrans, "monitor=0,bus=7,counter=1"); +PMU_EVENT_ATTR_STRING(argon_amax, bcm2711_argon_amax, "monitor=0,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(argon_wwait, bcm2711_argon_wwait, "monitor=0,bus=7,counter=3"); +PMU_EVENT_ATTR_STRING(argon_wtrans, bcm2711_argon_wtrans, "monitor=0,bus=7,counter=4"); +PMU_EVENT_ATTR_STRING(argon_wmax, bcm2711_argon_wmax, "monitor=0,bus=7,counter=5"); +PMU_EVENT_ATTR_STRING(argon_rwait, bcm2711_argon_rwait, "monitor=0,bus=7,counter=6"); +PMU_EVENT_ATTR_STRING(argon_rtrans, bcm2711_argon_rtrans, "monitor=0,bus=7,counter=7"); +PMU_EVENT_ATTR_STRING(argon_rmax, bcm2711_argon_rmax, "monitor=0,bus=7,counter=8"); +PMU_EVENT_ATTR_STRING(argon_rpend, bcm2711_argon_rpend, "monitor=0,bus=7,counter=9"); +PMU_EVENT_ATTR_STRING(h264_atwait, bcm2711_h264_atwait, "monitor=0,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(h264_atrans, bcm2711_h264_atrans, "monitor=0,bus=8,counter=1"); +PMU_EVENT_ATTR_STRING(h264_amax, bcm2711_h264_amax, "monitor=0,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(h264_wwait, bcm2711_h264_wwait, "monitor=0,bus=8,counter=3"); +PMU_EVENT_ATTR_STRING(h264_wtrans, bcm2711_h264_wtrans, "monitor=0,bus=8,counter=4"); +PMU_EVENT_ATTR_STRING(h264_wmax, bcm2711_h264_wmax, "monitor=0,bus=8,counter=5"); +PMU_EVENT_ATTR_STRING(h264_rwait, bcm2711_h264_rwait, "monitor=0,bus=8,counter=6"); +PMU_EVENT_ATTR_STRING(h264_rtrans, bcm2711_h264_rtrans, "monitor=0,bus=8,counter=7"); +PMU_EVENT_ATTR_STRING(h264_rmax, bcm2711_h264_rmax, "monitor=0,bus=8,counter=8"); +PMU_EVENT_ATTR_STRING(h264_rpend, bcm2711_h264_rpend, "monitor=0,bus=8,counter=9"); +PMU_EVENT_ATTR_STRING(peripheral_atwait, bcm2711_peripheral_atwait, "monitor=0,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(peripheral_atrans, bcm2711_peripheral_atrans, "monitor=0,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(peripheral_amax, bcm2711_peripheral_amax, "monitor=0,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(peripheral_wwait, bcm2711_peripheral_wwait, "monitor=0,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(peripheral_wtrans, bcm2711_peripheral_wtrans, "monitor=0,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(peripheral_wmax, bcm2711_peripheral_wmax, "monitor=0,bus=9,counter=5"); +PMU_EVENT_ATTR_STRING(peripheral_rwait, bcm2711_peripheral_rwait, "monitor=0,bus=9,counter=6"); +PMU_EVENT_ATTR_STRING(peripheral_rtrans, bcm2711_peripheral_rtrans, "monitor=0,bus=9,counter=7"); +PMU_EVENT_ATTR_STRING(peripheral_rmax, bcm2711_peripheral_rmax, "monitor=0,bus=9,counter=8"); +PMU_EVENT_ATTR_STRING(peripheral_rpend, bcm2711_peripheral_rpend, "monitor=0,bus=9,counter=9"); +PMU_EVENT_ATTR_STRING(arm_uc_atwait, bcm2711_arm_uc_atwait, "monitor=0,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(arm_uc_atrans, bcm2711_arm_uc_atrans, "monitor=0,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(arm_uc_amax, bcm2711_arm_uc_amax, "monitor=0,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(arm_uc_wwait, bcm2711_arm_uc_wwait, "monitor=0,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(arm_uc_wtrans, bcm2711_arm_uc_wtrans, "monitor=0,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(arm_uc_wmax, bcm2711_arm_uc_wmax, "monitor=0,bus=10,counter=5"); +PMU_EVENT_ATTR_STRING(arm_uc_rwait, bcm2711_arm_uc_rwait, "monitor=0,bus=10,counter=6"); +PMU_EVENT_ATTR_STRING(arm_uc_rtrans, bcm2711_arm_uc_rtrans, "monitor=0,bus=10,counter=7"); +PMU_EVENT_ATTR_STRING(arm_uc_rmax, bcm2711_arm_uc_rmax, "monitor=0,bus=10,counter=8"); +PMU_EVENT_ATTR_STRING(arm_uc_rpend, bcm2711_arm_uc_rpend, "monitor=0,bus=10,counter=9"); +PMU_EVENT_ATTR_STRING(arm_l2_atwait, bcm2711_arm_l2_atwait, "monitor=0,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(arm_l2_atrans, bcm2711_arm_l2_atrans, "monitor=0,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(arm_l2_amax, bcm2711_arm_l2_amax, "monitor=0,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(arm_l2_wwait, bcm2711_arm_l2_wwait, "monitor=0,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(arm_l2_wtrans, bcm2711_arm_l2_wtrans, "monitor=0,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(arm_l2_wmax, bcm2711_arm_l2_wmax, "monitor=0,bus=11,counter=5"); +PMU_EVENT_ATTR_STRING(arm_l2_rwait, bcm2711_arm_l2_rwait, "monitor=0,bus=11,counter=6"); +PMU_EVENT_ATTR_STRING(arm_l2_rtrans, bcm2711_arm_l2_rtrans, "monitor=0,bus=11,counter=7"); +PMU_EVENT_ATTR_STRING(arm_l2_rmax, bcm2711_arm_l2_rmax, "monitor=0,bus=11,counter=8"); +PMU_EVENT_ATTR_STRING(arm_l2_rpend, bcm2711_arm_l2_rpend, "monitor=0,bus=11,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_atwait, bcm2711_vpu_vpu1_d_l2_atwait, "monitor=1,bus=0,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_atrans, bcm2711_vpu_vpu1_d_l2_atrans, "monitor=1,bus=0,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_amax, bcm2711_vpu_vpu1_d_l2_amax, "monitor=1,bus=0,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_wwait, bcm2711_vpu_vpu1_d_l2_wwait, "monitor=1,bus=0,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_wtrans, bcm2711_vpu_vpu1_d_l2_wtrans, "monitor=1,bus=0,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_wmax, bcm2711_vpu_vpu1_d_l2_wmax, "monitor=1,bus=0,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rwait, bcm2711_vpu_vpu1_d_l2_rwait, "monitor=1,bus=0,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rtrans, bcm2711_vpu_vpu1_d_l2_rtrans, "monitor=1,bus=0,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rmax, bcm2711_vpu_vpu1_d_l2_rmax, "monitor=1,bus=0,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_l2_rpend, bcm2711_vpu_vpu1_d_l2_rpend, "monitor=1,bus=0,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_atwait, bcm2711_vpu_vpu0_d_l2_atwait, "monitor=1,bus=1,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_atrans, bcm2711_vpu_vpu0_d_l2_atrans, "monitor=1,bus=1,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_amax, bcm2711_vpu_vpu0_d_l2_amax, "monitor=1,bus=1,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_wwait, bcm2711_vpu_vpu0_d_l2_wwait, "monitor=1,bus=1,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_wtrans, bcm2711_vpu_vpu0_d_l2_wtrans, "monitor=1,bus=1,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_wmax, bcm2711_vpu_vpu0_d_l2_wmax, "monitor=1,bus=1,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rwait, bcm2711_vpu_vpu0_d_l2_rwait, "monitor=1,bus=1,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rtrans, bcm2711_vpu_vpu0_d_l2_rtrans, "monitor=1,bus=1,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rmax, bcm2711_vpu_vpu0_d_l2_rmax, "monitor=1,bus=1,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_l2_rpend, bcm2711_vpu_vpu0_d_l2_rpend, "monitor=1,bus=1,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_atwait, bcm2711_vpu_vpu1_i_l2_atwait, "monitor=1,bus=2,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_atrans, bcm2711_vpu_vpu1_i_l2_atrans, "monitor=1,bus=2,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_amax, bcm2711_vpu_vpu1_i_l2_amax, "monitor=1,bus=2,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_wwait, bcm2711_vpu_vpu1_i_l2_wwait, "monitor=1,bus=2,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_wtrans, bcm2711_vpu_vpu1_i_l2_wtrans, "monitor=1,bus=2,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_wmax, bcm2711_vpu_vpu1_i_l2_wmax, "monitor=1,bus=2,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rwait, bcm2711_vpu_vpu1_i_l2_rwait, "monitor=1,bus=2,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rtrans, bcm2711_vpu_vpu1_i_l2_rtrans, "monitor=1,bus=2,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rmax, bcm2711_vpu_vpu1_i_l2_rmax, "monitor=1,bus=2,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_l2_rpend, bcm2711_vpu_vpu1_i_l2_rpend, "monitor=1,bus=2,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_atwait, bcm2711_vpu_vpu0_i_l2_atwait, "monitor=1,bus=3,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_atrans, bcm2711_vpu_vpu0_i_l2_atrans, "monitor=1,bus=3,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_amax, bcm2711_vpu_vpu0_i_l2_amax, "monitor=1,bus=3,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_wwait, bcm2711_vpu_vpu0_i_l2_wwait, "monitor=1,bus=3,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_wtrans, bcm2711_vpu_vpu0_i_l2_wtrans, "monitor=1,bus=3,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_wmax, bcm2711_vpu_vpu0_i_l2_wmax, "monitor=1,bus=3,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rwait, bcm2711_vpu_vpu0_i_l2_rwait, "monitor=1,bus=3,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rtrans, bcm2711_vpu_vpu0_i_l2_rtrans, "monitor=1,bus=3,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rmax, bcm2711_vpu_vpu0_i_l2_rmax, "monitor=1,bus=3,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_l2_rpend, bcm2711_vpu_vpu0_i_l2_rpend, "monitor=1,bus=3,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_atwait, bcm2711_vpu_system_l2_atwait, "monitor=1,bus=4,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, bcm2711_vpu_system_l2_atrans, "monitor=1,bus=4,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_amax, bcm2711_vpu_system_l2_amax, "monitor=1,bus=4,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wwait, bcm2711_vpu_system_l2_wwait, "monitor=1,bus=4,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, bcm2711_vpu_system_l2_wtrans, "monitor=1,bus=4,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wmax, bcm2711_vpu_system_l2_wmax, "monitor=1,bus=4,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rwait, bcm2711_vpu_system_l2_rwait, "monitor=1,bus=4,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, bcm2711_vpu_system_l2_rtrans, "monitor=1,bus=4,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rmax, bcm2711_vpu_system_l2_rmax, "monitor=1,bus=4,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rpend, bcm2711_vpu_system_l2_rpend, "monitor=1,bus=4,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atwait, bcm2711_vpu_dma_l2_atwait, "monitor=1,bus=5,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, bcm2711_vpu_dma_l2_atrans, "monitor=1,bus=5,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_amax, bcm2711_vpu_dma_l2_amax, "monitor=1,bus=5,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_wwait, bcm2711_vpu_dma_l2_wwait, "monitor=1,bus=5,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_wtrans, bcm2711_vpu_dma_l2_wtrans, "monitor=1,bus=5,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_wmax, bcm2711_vpu_dma_l2_wmax, "monitor=1,bus=5,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rwait, bcm2711_vpu_dma_l2_rwait, "monitor=1,bus=5,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rtrans, bcm2711_vpu_dma_l2_rtrans, "monitor=1,bus=5,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rmax, bcm2711_vpu_dma_l2_rmax, "monitor=1,bus=5,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_rpend, bcm2711_vpu_dma_l2_rpend, "monitor=1,bus=5,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_atwait, bcm2711_vpu_vpu1_d_uc_atwait, "monitor=1,bus=6,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_atrans, bcm2711_vpu_vpu1_d_uc_atrans, "monitor=1,bus=6,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_amax, bcm2711_vpu_vpu1_d_uc_amax, "monitor=1,bus=6,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_wwait, bcm2711_vpu_vpu1_d_uc_wwait, "monitor=1,bus=6,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_wtrans, bcm2711_vpu_vpu1_d_uc_wtrans, "monitor=1,bus=6,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_wmax, bcm2711_vpu_vpu1_d_uc_wmax, "monitor=1,bus=6,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rwait, bcm2711_vpu_vpu1_d_uc_rwait, "monitor=1,bus=6,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rtrans, bcm2711_vpu_vpu1_d_uc_rtrans, "monitor=1,bus=6,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rmax, bcm2711_vpu_vpu1_d_uc_rmax, "monitor=1,bus=6,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_d_uc_rpend, bcm2711_vpu_vpu1_d_uc_rpend, "monitor=1,bus=6,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_atwait, bcm2711_vpu_vpu0_d_uc_atwait, "monitor=1,bus=7,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_atrans, bcm2711_vpu_vpu0_d_uc_atrans, "monitor=1,bus=7,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_amax, bcm2711_vpu_vpu0_d_uc_amax, "monitor=1,bus=7,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_wwait, bcm2711_vpu_vpu0_d_uc_wwait, "monitor=1,bus=7,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_wtrans, bcm2711_vpu_vpu0_d_uc_wtrans, "monitor=1,bus=7,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_wmax, bcm2711_vpu_vpu0_d_uc_wmax, "monitor=1,bus=7,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rwait, bcm2711_vpu_vpu0_d_uc_rwait, "monitor=1,bus=7,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rtrans, bcm2711_vpu_vpu0_d_uc_rtrans, "monitor=1,bus=7,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rmax, bcm2711_vpu_vpu0_d_uc_rmax, "monitor=1,bus=7,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_d_uc_rpend, bcm2711_vpu_vpu0_d_uc_rpend, "monitor=1,bus=7,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_atwait, bcm2711_vpu_vpu1_i_uc_atwait, "monitor=1,bus=8,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_atrans, bcm2711_vpu_vpu1_i_uc_atrans, "monitor=1,bus=8,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_amax, bcm2711_vpu_vpu1_i_uc_amax, "monitor=1,bus=8,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_wwait, bcm2711_vpu_vpu1_i_uc_wwait, "monitor=1,bus=8,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_wtrans, bcm2711_vpu_vpu1_i_uc_wtrans, "monitor=1,bus=8,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_wmax, bcm2711_vpu_vpu1_i_uc_wmax, "monitor=1,bus=8,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rwait, bcm2711_vpu_vpu1_i_uc_rwait, "monitor=1,bus=8,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rtrans, bcm2711_vpu_vpu1_i_uc_rtrans, "monitor=1,bus=8,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rmax, bcm2711_vpu_vpu1_i_uc_rmax, "monitor=1,bus=8,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu1_i_uc_rpend, bcm2711_vpu_vpu1_i_uc_rpend, "monitor=1,bus=8,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_atwait, bcm2711_vpu_vpu0_i_uc_atwait, "monitor=1,bus=9,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_atrans, bcm2711_vpu_vpu0_i_uc_atrans, "monitor=1,bus=9,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_amax, bcm2711_vpu_vpu0_i_uc_amax, "monitor=1,bus=9,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_wwait, bcm2711_vpu_vpu0_i_uc_wwait, "monitor=1,bus=9,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_wtrans, bcm2711_vpu_vpu0_i_uc_wtrans, "monitor=1,bus=9,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_wmax, bcm2711_vpu_vpu0_i_uc_wmax, "monitor=1,bus=9,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rwait, bcm2711_vpu_vpu0_i_uc_rwait, "monitor=1,bus=9,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rtrans, bcm2711_vpu_vpu0_i_uc_rtrans, "monitor=1,bus=9,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rmax, bcm2711_vpu_vpu0_i_uc_rmax, "monitor=1,bus=9,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu0_i_uc_rpend, bcm2711_vpu_vpu0_i_uc_rpend, "monitor=1,bus=9,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_atwait, bcm2711_vpu_vpu_uc_atwait, "monitor=1,bus=10,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_atrans, bcm2711_vpu_vpu_uc_atrans, "monitor=1,bus=10,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_amax, bcm2711_vpu_vpu_uc_amax, "monitor=1,bus=10,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_wwait, bcm2711_vpu_vpu_uc_wwait, "monitor=1,bus=10,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_wtrans, bcm2711_vpu_vpu_uc_wtrans, "monitor=1,bus=10,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_wmax, bcm2711_vpu_vpu_uc_wmax, "monitor=1,bus=10,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rwait, bcm2711_vpu_vpu_uc_rwait, "monitor=1,bus=10,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rtrans, bcm2711_vpu_vpu_uc_rtrans, "monitor=1,bus=10,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rmax, bcm2711_vpu_vpu_uc_rmax, "monitor=1,bus=10,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_vpu_uc_rpend, bcm2711_vpu_vpu_uc_rpend, "monitor=1,bus=10,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_atwait, bcm2711_vpu_l2_out_atwait, "monitor=1,bus=11,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, bcm2711_vpu_l2_out_atrans, "monitor=1,bus=11,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_amax, bcm2711_vpu_l2_out_amax, "monitor=1,bus=11,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_wwait, bcm2711_vpu_l2_out_wwait, "monitor=1,bus=11,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_wtrans, bcm2711_vpu_l2_out_wtrans, "monitor=1,bus=11,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_wmax, bcm2711_vpu_l2_out_wmax, "monitor=1,bus=11,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rwait, bcm2711_vpu_l2_out_rwait, "monitor=1,bus=11,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rtrans, bcm2711_vpu_l2_out_rtrans, "monitor=1,bus=11,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rmax, bcm2711_vpu_l2_out_rmax, "monitor=1,bus=11,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_l2_out_rpend, bcm2711_vpu_l2_out_rpend, "monitor=1,bus=11,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atwait, bcm2711_vpu_dma_uc_atwait, "monitor=1,bus=12,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, bcm2711_vpu_dma_uc_atrans, "monitor=1,bus=12,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_amax, bcm2711_vpu_dma_uc_amax, "monitor=1,bus=12,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_wwait, bcm2711_vpu_dma_uc_wwait, "monitor=1,bus=12,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_wtrans, bcm2711_vpu_dma_uc_wtrans, "monitor=1,bus=12,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_wmax, bcm2711_vpu_dma_uc_wmax, "monitor=1,bus=12,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rwait, bcm2711_vpu_dma_uc_rwait, "monitor=1,bus=12,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rtrans, bcm2711_vpu_dma_uc_rtrans, "monitor=1,bus=12,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rmax, bcm2711_vpu_dma_uc_rmax, "monitor=1,bus=12,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_rpend, bcm2711_vpu_dma_uc_rpend, "monitor=1,bus=12,counter=9"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_atwait, bcm2711_vpu_l2_in_atwait, "monitor=1,bus=13,counter=0"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, bcm2711_vpu_l2_in_atrans, "monitor=1,bus=13,counter=1"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_amax, bcm2711_vpu_l2_in_amax, "monitor=1,bus=13,counter=2"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_wwait, bcm2711_vpu_l2_in_wwait, "monitor=1,bus=13,counter=3"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_wtrans, bcm2711_vpu_l2_in_wtrans, "monitor=1,bus=13,counter=4"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_wmax, bcm2711_vpu_l2_in_wmax, "monitor=1,bus=13,counter=5"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rwait, bcm2711_vpu_l2_in_rwait, "monitor=1,bus=13,counter=6"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rtrans, bcm2711_vpu_l2_in_rtrans, "monitor=1,bus=13,counter=7"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rmax, bcm2711_vpu_l2_in_rmax, "monitor=1,bus=13,counter=8"); +PMU_EVENT_ATTR_STRING(vpu_l2_in_rpend, bcm2711_vpu_l2_in_rpend, "monitor=1,bus=13,counter=9"); + +static struct attribute *bcm2711_events[] = { + &bcm2711_dma_l2_atwait.attr.attr, + &bcm2711_dma_l2_atrans.attr.attr, + &bcm2711_dma_l2_amax.attr.attr, + &bcm2711_dma_l2_wwait.attr.attr, + &bcm2711_dma_l2_wtrans.attr.attr, + &bcm2711_dma_l2_wmax.attr.attr, + &bcm2711_dma_l2_rwait.attr.attr, + &bcm2711_dma_l2_rtrans.attr.attr, + &bcm2711_dma_l2_rmax.attr.attr, + &bcm2711_dma_l2_rpend.attr.attr, + &bcm2711_trans_atwait.attr.attr, + &bcm2711_trans_atrans.attr.attr, + &bcm2711_trans_amax.attr.attr, + &bcm2711_trans_wwait.attr.attr, + &bcm2711_trans_wtrans.attr.attr, + &bcm2711_trans_wmax.attr.attr, + &bcm2711_trans_rwait.attr.attr, + &bcm2711_trans_rtrans.attr.attr, + &bcm2711_trans_rmax.attr.attr, + &bcm2711_trans_rpend.attr.attr, + &bcm2711_jpeg_atwait.attr.attr, + &bcm2711_jpeg_atrans.attr.attr, + &bcm2711_jpeg_amax.attr.attr, + &bcm2711_jpeg_wwait.attr.attr, + &bcm2711_jpeg_wtrans.attr.attr, + &bcm2711_jpeg_wmax.attr.attr, + &bcm2711_jpeg_rwait.attr.attr, + &bcm2711_jpeg_rtrans.attr.attr, + &bcm2711_jpeg_rmax.attr.attr, + &bcm2711_jpeg_rpend.attr.attr, + &bcm2711_vpu_uc_atwait.attr.attr, + &bcm2711_vpu_uc_atrans.attr.attr, + &bcm2711_vpu_uc_amax.attr.attr, + &bcm2711_vpu_uc_wwait.attr.attr, + &bcm2711_vpu_uc_wtrans.attr.attr, + &bcm2711_vpu_uc_wmax.attr.attr, + &bcm2711_vpu_uc_rwait.attr.attr, + &bcm2711_vpu_uc_rtrans.attr.attr, + &bcm2711_vpu_uc_rmax.attr.attr, + &bcm2711_vpu_uc_rpend.attr.attr, + &bcm2711_dma_uc_atwait.attr.attr, + &bcm2711_dma_uc_atrans.attr.attr, + &bcm2711_dma_uc_amax.attr.attr, + &bcm2711_dma_uc_wwait.attr.attr, + &bcm2711_dma_uc_wtrans.attr.attr, + &bcm2711_dma_uc_wmax.attr.attr, + &bcm2711_dma_uc_rwait.attr.attr, + &bcm2711_dma_uc_rtrans.attr.attr, + &bcm2711_dma_uc_rmax.attr.attr, + &bcm2711_dma_uc_rpend.attr.attr, + &bcm2711_system_l2_atwait.attr.attr, + &bcm2711_system_l2_atrans.attr.attr, + &bcm2711_system_l2_amax.attr.attr, + &bcm2711_system_l2_wwait.attr.attr, + &bcm2711_system_l2_wtrans.attr.attr, + &bcm2711_system_l2_wmax.attr.attr, + &bcm2711_system_l2_rwait.attr.attr, + &bcm2711_system_l2_rtrans.attr.attr, + &bcm2711_system_l2_rmax.attr.attr, + &bcm2711_system_l2_rpend.attr.attr, + &bcm2711_hvs_atwait.attr.attr, + &bcm2711_hvs_atrans.attr.attr, + &bcm2711_hvs_amax.attr.attr, + &bcm2711_hvs_wwait.attr.attr, + &bcm2711_hvs_wtrans.attr.attr, + &bcm2711_hvs_wmax.attr.attr, + &bcm2711_hvs_rwait.attr.attr, + &bcm2711_hvs_rtrans.attr.attr, + &bcm2711_hvs_rmax.attr.attr, + &bcm2711_hvs_rpend.attr.attr, + &bcm2711_argon_atwait.attr.attr, + &bcm2711_argon_atrans.attr.attr, + &bcm2711_argon_amax.attr.attr, + &bcm2711_argon_wwait.attr.attr, + &bcm2711_argon_wtrans.attr.attr, + &bcm2711_argon_wmax.attr.attr, + &bcm2711_argon_rwait.attr.attr, + &bcm2711_argon_rtrans.attr.attr, + &bcm2711_argon_rmax.attr.attr, + &bcm2711_argon_rpend.attr.attr, + &bcm2711_h264_atwait.attr.attr, + &bcm2711_h264_atrans.attr.attr, + &bcm2711_h264_amax.attr.attr, + &bcm2711_h264_wwait.attr.attr, + &bcm2711_h264_wtrans.attr.attr, + &bcm2711_h264_wmax.attr.attr, + &bcm2711_h264_rwait.attr.attr, + &bcm2711_h264_rtrans.attr.attr, + &bcm2711_h264_rmax.attr.attr, + &bcm2711_h264_rpend.attr.attr, + &bcm2711_peripheral_atwait.attr.attr, + &bcm2711_peripheral_atrans.attr.attr, + &bcm2711_peripheral_amax.attr.attr, + &bcm2711_peripheral_wwait.attr.attr, + &bcm2711_peripheral_wtrans.attr.attr, + &bcm2711_peripheral_wmax.attr.attr, + &bcm2711_peripheral_rwait.attr.attr, + &bcm2711_peripheral_rtrans.attr.attr, + &bcm2711_peripheral_rmax.attr.attr, + &bcm2711_peripheral_rpend.attr.attr, + &bcm2711_arm_uc_atwait.attr.attr, + &bcm2711_arm_uc_atrans.attr.attr, + &bcm2711_arm_uc_amax.attr.attr, + &bcm2711_arm_uc_wwait.attr.attr, + &bcm2711_arm_uc_wtrans.attr.attr, + &bcm2711_arm_uc_wmax.attr.attr, + &bcm2711_arm_uc_rwait.attr.attr, + &bcm2711_arm_uc_rtrans.attr.attr, + &bcm2711_arm_uc_rmax.attr.attr, + &bcm2711_arm_uc_rpend.attr.attr, + &bcm2711_arm_l2_atwait.attr.attr, + &bcm2711_arm_l2_atrans.attr.attr, + &bcm2711_arm_l2_amax.attr.attr, + &bcm2711_arm_l2_wwait.attr.attr, + &bcm2711_arm_l2_wtrans.attr.attr, + &bcm2711_arm_l2_wmax.attr.attr, + &bcm2711_arm_l2_rwait.attr.attr, + &bcm2711_arm_l2_rtrans.attr.attr, + &bcm2711_arm_l2_rmax.attr.attr, + &bcm2711_arm_l2_rpend.attr.attr, + &bcm2711_vpu_vpu1_d_l2_atwait.attr.attr, + &bcm2711_vpu_vpu1_d_l2_atrans.attr.attr, + &bcm2711_vpu_vpu1_d_l2_amax.attr.attr, + &bcm2711_vpu_vpu1_d_l2_wwait.attr.attr, + &bcm2711_vpu_vpu1_d_l2_wtrans.attr.attr, + &bcm2711_vpu_vpu1_d_l2_wmax.attr.attr, + &bcm2711_vpu_vpu1_d_l2_rwait.attr.attr, + &bcm2711_vpu_vpu1_d_l2_rtrans.attr.attr, + &bcm2711_vpu_vpu1_d_l2_rmax.attr.attr, + &bcm2711_vpu_vpu1_d_l2_rpend.attr.attr, + &bcm2711_vpu_vpu0_d_l2_atwait.attr.attr, + &bcm2711_vpu_vpu0_d_l2_atrans.attr.attr, + &bcm2711_vpu_vpu0_d_l2_amax.attr.attr, + &bcm2711_vpu_vpu0_d_l2_wwait.attr.attr, + &bcm2711_vpu_vpu0_d_l2_wtrans.attr.attr, + &bcm2711_vpu_vpu0_d_l2_wmax.attr.attr, + &bcm2711_vpu_vpu0_d_l2_rwait.attr.attr, + &bcm2711_vpu_vpu0_d_l2_rtrans.attr.attr, + &bcm2711_vpu_vpu0_d_l2_rmax.attr.attr, + &bcm2711_vpu_vpu0_d_l2_rpend.attr.attr, + &bcm2711_vpu_vpu1_i_l2_atwait.attr.attr, + &bcm2711_vpu_vpu1_i_l2_atrans.attr.attr, + &bcm2711_vpu_vpu1_i_l2_amax.attr.attr, + &bcm2711_vpu_vpu1_i_l2_wwait.attr.attr, + &bcm2711_vpu_vpu1_i_l2_wtrans.attr.attr, + &bcm2711_vpu_vpu1_i_l2_wmax.attr.attr, + &bcm2711_vpu_vpu1_i_l2_rwait.attr.attr, + &bcm2711_vpu_vpu1_i_l2_rtrans.attr.attr, + &bcm2711_vpu_vpu1_i_l2_rmax.attr.attr, + &bcm2711_vpu_vpu1_i_l2_rpend.attr.attr, + &bcm2711_vpu_vpu0_i_l2_atwait.attr.attr, + &bcm2711_vpu_vpu0_i_l2_atrans.attr.attr, + &bcm2711_vpu_vpu0_i_l2_amax.attr.attr, + &bcm2711_vpu_vpu0_i_l2_wwait.attr.attr, + &bcm2711_vpu_vpu0_i_l2_wtrans.attr.attr, + &bcm2711_vpu_vpu0_i_l2_wmax.attr.attr, + &bcm2711_vpu_vpu0_i_l2_rwait.attr.attr, + &bcm2711_vpu_vpu0_i_l2_rtrans.attr.attr, + &bcm2711_vpu_vpu0_i_l2_rmax.attr.attr, + &bcm2711_vpu_vpu0_i_l2_rpend.attr.attr, + &bcm2711_vpu_system_l2_atwait.attr.attr, + &bcm2711_vpu_system_l2_atrans.attr.attr, + &bcm2711_vpu_system_l2_amax.attr.attr, + &bcm2711_vpu_system_l2_wwait.attr.attr, + &bcm2711_vpu_system_l2_wtrans.attr.attr, + &bcm2711_vpu_system_l2_wmax.attr.attr, + &bcm2711_vpu_system_l2_rwait.attr.attr, + &bcm2711_vpu_system_l2_rtrans.attr.attr, + &bcm2711_vpu_system_l2_rmax.attr.attr, + &bcm2711_vpu_system_l2_rpend.attr.attr, + &bcm2711_vpu_dma_l2_atwait.attr.attr, + &bcm2711_vpu_dma_l2_atrans.attr.attr, + &bcm2711_vpu_dma_l2_amax.attr.attr, + &bcm2711_vpu_dma_l2_wwait.attr.attr, + &bcm2711_vpu_dma_l2_wtrans.attr.attr, + &bcm2711_vpu_dma_l2_wmax.attr.attr, + &bcm2711_vpu_dma_l2_rwait.attr.attr, + &bcm2711_vpu_dma_l2_rtrans.attr.attr, + &bcm2711_vpu_dma_l2_rmax.attr.attr, + &bcm2711_vpu_dma_l2_rpend.attr.attr, + &bcm2711_vpu_vpu1_d_uc_atwait.attr.attr, + &bcm2711_vpu_vpu1_d_uc_atrans.attr.attr, + &bcm2711_vpu_vpu1_d_uc_amax.attr.attr, + &bcm2711_vpu_vpu1_d_uc_wwait.attr.attr, + &bcm2711_vpu_vpu1_d_uc_wtrans.attr.attr, + &bcm2711_vpu_vpu1_d_uc_wmax.attr.attr, + &bcm2711_vpu_vpu1_d_uc_rwait.attr.attr, + &bcm2711_vpu_vpu1_d_uc_rtrans.attr.attr, + &bcm2711_vpu_vpu1_d_uc_rmax.attr.attr, + &bcm2711_vpu_vpu1_d_uc_rpend.attr.attr, + &bcm2711_vpu_vpu0_d_uc_atwait.attr.attr, + &bcm2711_vpu_vpu0_d_uc_atrans.attr.attr, + &bcm2711_vpu_vpu0_d_uc_amax.attr.attr, + &bcm2711_vpu_vpu0_d_uc_wwait.attr.attr, + &bcm2711_vpu_vpu0_d_uc_wtrans.attr.attr, + &bcm2711_vpu_vpu0_d_uc_wmax.attr.attr, + &bcm2711_vpu_vpu0_d_uc_rwait.attr.attr, + &bcm2711_vpu_vpu0_d_uc_rtrans.attr.attr, + &bcm2711_vpu_vpu0_d_uc_rmax.attr.attr, + &bcm2711_vpu_vpu0_d_uc_rpend.attr.attr, + &bcm2711_vpu_vpu1_i_uc_atwait.attr.attr, + &bcm2711_vpu_vpu1_i_uc_atrans.attr.attr, + &bcm2711_vpu_vpu1_i_uc_amax.attr.attr, + &bcm2711_vpu_vpu1_i_uc_wwait.attr.attr, + &bcm2711_vpu_vpu1_i_uc_wtrans.attr.attr, + &bcm2711_vpu_vpu1_i_uc_wmax.attr.attr, + &bcm2711_vpu_vpu1_i_uc_rwait.attr.attr, + &bcm2711_vpu_vpu1_i_uc_rtrans.attr.attr, + &bcm2711_vpu_vpu1_i_uc_rmax.attr.attr, + &bcm2711_vpu_vpu1_i_uc_rpend.attr.attr, + &bcm2711_vpu_vpu0_i_uc_atwait.attr.attr, + &bcm2711_vpu_vpu0_i_uc_atrans.attr.attr, + &bcm2711_vpu_vpu0_i_uc_amax.attr.attr, + &bcm2711_vpu_vpu0_i_uc_wwait.attr.attr, + &bcm2711_vpu_vpu0_i_uc_wtrans.attr.attr, + &bcm2711_vpu_vpu0_i_uc_wmax.attr.attr, + &bcm2711_vpu_vpu0_i_uc_rwait.attr.attr, + &bcm2711_vpu_vpu0_i_uc_rtrans.attr.attr, + &bcm2711_vpu_vpu0_i_uc_rmax.attr.attr, + &bcm2711_vpu_vpu0_i_uc_rpend.attr.attr, + &bcm2711_vpu_vpu_uc_atwait.attr.attr, + &bcm2711_vpu_vpu_uc_atrans.attr.attr, + &bcm2711_vpu_vpu_uc_amax.attr.attr, + &bcm2711_vpu_vpu_uc_wwait.attr.attr, + &bcm2711_vpu_vpu_uc_wtrans.attr.attr, + &bcm2711_vpu_vpu_uc_wmax.attr.attr, + &bcm2711_vpu_vpu_uc_rwait.attr.attr, + &bcm2711_vpu_vpu_uc_rtrans.attr.attr, + &bcm2711_vpu_vpu_uc_rmax.attr.attr, + &bcm2711_vpu_vpu_uc_rpend.attr.attr, + &bcm2711_vpu_l2_out_atwait.attr.attr, + &bcm2711_vpu_l2_out_atrans.attr.attr, + &bcm2711_vpu_l2_out_amax.attr.attr, + &bcm2711_vpu_l2_out_wwait.attr.attr, + &bcm2711_vpu_l2_out_wtrans.attr.attr, + &bcm2711_vpu_l2_out_wmax.attr.attr, + &bcm2711_vpu_l2_out_rwait.attr.attr, + &bcm2711_vpu_l2_out_rtrans.attr.attr, + &bcm2711_vpu_l2_out_rmax.attr.attr, + &bcm2711_vpu_l2_out_rpend.attr.attr, + &bcm2711_vpu_dma_uc_atwait.attr.attr, + &bcm2711_vpu_dma_uc_atrans.attr.attr, + &bcm2711_vpu_dma_uc_amax.attr.attr, + &bcm2711_vpu_dma_uc_wwait.attr.attr, + &bcm2711_vpu_dma_uc_wtrans.attr.attr, + &bcm2711_vpu_dma_uc_wmax.attr.attr, + &bcm2711_vpu_dma_uc_rwait.attr.attr, + &bcm2711_vpu_dma_uc_rtrans.attr.attr, + &bcm2711_vpu_dma_uc_rmax.attr.attr, + &bcm2711_vpu_dma_uc_rpend.attr.attr, + &bcm2711_vpu_l2_in_atwait.attr.attr, + &bcm2711_vpu_l2_in_atrans.attr.attr, + &bcm2711_vpu_l2_in_amax.attr.attr, + &bcm2711_vpu_l2_in_wwait.attr.attr, + &bcm2711_vpu_l2_in_wtrans.attr.attr, + &bcm2711_vpu_l2_in_wmax.attr.attr, + &bcm2711_vpu_l2_in_rwait.attr.attr, + &bcm2711_vpu_l2_in_rtrans.attr.attr, + &bcm2711_vpu_l2_in_rmax.attr.attr, + &bcm2711_vpu_l2_in_rpend.attr.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_bcm2711_events_group = { + .name = "events", + .attrs = bcm2711_events, + .is_visible = rpi_axi_pmu_events_is_visible, +}; + +static const struct attribute_group *rpi_axi_pmu_bcm2835_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_bcm2835_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +static const struct attribute_group *rpi_axi_pmu_bcm2711_attr_groups[] = { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_bcm2711_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capability + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight check + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON_MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu != pmu) + return false; + mon = config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event) >= 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON_MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON_SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON_VPU]); + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + pmu = pmu_to_rpi_axi_pmu(event->pmu); + dev = pmu->pmu.dev; + if (!config_is_valid(pmu, event->attr.config) || + event->attr.config1 || event->attr.config2) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0 || event->attach_state & PERF_ATTACH_TASK) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu != pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx = -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON_SYSTEM or MON_VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon, u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + __le32 tmp[3] = { + cpu_to_le32(pmu->monitor[mon].mailbox + GEN_CTRL), + cpu_to_le32(1), + cpu_to_le32(set), + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || le32_to_cpu(tmp[1]) != 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(int idx) +{ + return BW0_CTRL + idx * BW_STRIDE; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, u32 set) +{ + int watcher = watcher_offset(idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + __le32 tmp[3] = { + cpu_to_le32(pmu->monitor[mon].mailbox + watcher), + cpu_to_le32(1), + cpu_to_le32(set), + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + err = rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || le32_to_cpu(tmp[1]) != 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (READ_ONCE(pmu->monitor[mon].hw_events.enabled[idx])) + return; + bus_control = BW_CTRL_ENABLE_BIT | + ((bus << BW_CTRL_BUS_WATCH_SHIFT) & BW_CTRL_BUS_WATCH_MASK); + if (filter) { + bus_control |= BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |= FIELD_PREP(BW_CTRL_BUS_FILTER_MASK, filter); + } + if (!pmu->monitor[mon].hw_events.monitor_running) { + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum monitor mon, int idx) +{ + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT_ATRANS: return BW_ATRANS_OFFSET; + case CNT_ATWAIT: return BW_ATWAIT_OFFSET; + case CNT_AMAX: return BW_AMAX_OFFSET; + case CNT_WTRANS: return BW_WTRANS_OFFSET; + case CNT_WWAIT: return BW_WTWAIT_OFFSET; + case CNT_WMAX: return BW_WMAX_OFFSET; + case CNT_RTRANS: return BW_RTRANS_OFFSET; + case CNT_RWAIT: return BW_RTWAIT_OFFSET; + case CNT_RMAX: return BW_RMAX_OFFSET; + case CNT_RATRANS: return BW_RATRANS_OFFSET; + case CNT_RPEND: return BW_RPEND_OFFSET; + default: return 0; + } +} + +static u32 counter_width_mask(enum counter counter) +{ + switch (counter) { + case CNT_ATWAIT: + case CNT_AMAX: + case CNT_WWAIT: + case CNT_WMAX: + case CNT_RWAIT: + case CNT_RMAX: + return GENMASK(27, 0); + case CNT_RPEND: + return GENMASK(7, 0); + default: + return U32_MAX; + } +} + +/* + * The max latency (CNT_AMAX, CNT_WMAX, CNT_RMAX) and pending read (CNT_RPEND) + * registers hold a hardware level/watermark rather than a running transaction + * or cycle count: + * - CNT_RPEND is an instantaneous register holding the number of reads + * currently pending at the time of the register read. + * - CNT_AMAX, CNT_WMAX, and CNT_RMAX are sticky hardware peak-hold registers + * that retain the maximum latency observed since the bus watcher was started + * (pulsing BW_CTRL_RESET_BIT on read would reset concurrent transaction and + * wait counters sharing the same bus watcher). + * + * Because userspace perf tools (such as 'perf stat' and 'perf stat -I') treat + * event->count as a monotonically non-decreasing cumulative counter and compute + * per-interval values by subtracting the event->count value read at the end of + * the previous interval, we cannot simply overwrite event->count with the raw + * register value (which would underflow u64 when the level decreases between + * intervals, or report 0 when the peak latency stays constant) nor add the raw + * level on every 100ms timer poll. + * + * Instead, for level counters, event->hw.prev_count tracks the maximum level + * observed since the last userspace read (initialized to 0 at event init and + * reset to 0 in rpi_axi_pmu_read()). Whenever a sample exceeds prev_count, + * only the increment (new_count - prev_count) is added to event->count. Thus, + * across any read interval starting at baseline C, event->count equals + * C + max(samples in interval), and subtracting the previous read's baseline C + * yields the maximum level observed during that interval (per-interval peak + * for CNT_RPEND, and current hardware watermark for CNT_AMAX/WMAX/RMAX). + */ +static bool counter_is_level(enum counter counter) +{ + return counter == CNT_AMAX || counter == CNT_WMAX || + counter == CNT_RMAX || counter == CNT_RPEND; +} + +static void rpi_axi_pmu_event_update(struct perf_event *event, u32 new_count) +{ + enum counter counter = config_to_counter(event->attr.config); + u32 mask = counter_width_mask(counter); + u32 prev_count = local64_read(&event->hw.prev_count); + + if (counter_is_level(counter)) { + new_count &= mask; + if (new_count > prev_count) { + local64_set(&event->hw.prev_count, new_count); + local64_add(new_count - prev_count, &event->count); + } + } else { + local64_set(&event->hw.prev_count, new_count); + local64_add((new_count - prev_count) & mask, &event->count); + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from watcher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor mon, int idx, + enum counter counter) +{ + int watcher = watcher_offset(idx); + int offset = counter_offset(counter); + + if (!READ_ONCE(pmu->monitor[mon].hw_events.enabled[idx])) + return 0; + lockdep_assert_held(&pmu->lock); + return readl(pmu->monitor[mon].base_address + watcher + offset); +} + +/* All three VPU bus watchers, from BW0_CTRL to BW2_RATRANS, in one mailbox call */ +#define VPU_READ_WORDS ((BW2_CTRL + BW_RATRANS_OFFSET - BW0_CTRL) / 4 + 1) + +static int rpi_axi_pmu_vpu_read_watchers(struct rpi_axi_pmu *pmu, u32 *regs) +{ + __le32 tmp[2 + VPU_READ_WORDS] = { + cpu_to_le32(pmu->monitor[MON_VPU].mailbox + BW0_CTRL), + cpu_to_le32(VPU_READ_WORDS), + }; + int err; + + lockdep_assert_held(&pmu->vpu_mutex); + err = rpi_firmware_property(pmu->firmware, RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err || le32_to_cpu(tmp[1]) != VPU_READ_WORDS) { + dev_err_ratelimited(&pmu->pdev->dev, "Failed to read bus watchers\n"); + return -EIO; + } + for (int i = 0; i < VPU_READ_WORDS; i++) + regs[i] = le32_to_cpu(tmp[2 + i]); + return 0; +} + +/** + * rpi_axi_pmu_update() - Samples hardware counter and updates event->count + * @event: Pointer to perf_event being updated + * @reset_level: True on userspace read() to reset the level counter baseline + * + * Thread-safe counter update implementation: + * - System Monitor (MON_SYSTEM): Uses fast MMIO (~15ns), serialized via pmu->lock spinlock. + * - VPU Monitor (MON_VPU): Async background polling in process context (vpu_work). + */ +static void rpi_axi_pmu_update(struct perf_event *event, bool reset_level) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long flags; + + raw_spin_lock_irqsave(&pmu->lock, flags); + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System) are read synchronously. + */ + if (!pmu->monitor[mon].use_mailbox_interface && + event->hw.idx >= 0 && + pmu->monitor[mon].hw_events.enabled[event->hw.idx] && + !(event->hw.state & PERF_HES_STOPPED) && + (event->hw.state & PERF_HES_UPTODATE)) { + u32 new_count = rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + + rpi_axi_pmu_event_update(event, new_count); + } + if (reset_level && counter_is_level(counter)) + local64_set(&event->hw.prev_count, 0); + raw_spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value on userspace read + * @event: Pointer to perf_event being read + * + * Updates the event count and, for level counters, resets event->hw.prev_count + * to 0 under pmu->lock so the current event->count serves as the baseline for + * the next read interval. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + rpi_axi_pmu_update(event, true); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monitor counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in process context under vpu_mutex. Programs newly allocated VPU bus + * watchers, then reads all of them in a single mailbox call and updates the + * VPU events. A watcher reassigned while the lock was dropped is detected by + * its config_gen and skipped. A newly started event takes its first read as + * its baseline (PERF_HES_UPTODATE). + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu = container_of(work, struct rpi_axi_pmu, vpu_work); + struct rpi_axi_hw_events *hw = &pmu->monitor[MON_VPU].hw_events; + unsigned int gen[NUM_BUS_WATCHERS_PER_MONITOR]; + bool valid[NUM_BUS_WATCHERS_PER_MONITOR]; + u32 regs[VPU_READ_WORDS]; + bool any = false; + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + raw_spin_lock_irq(&pmu->lock); + for (int idx = 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) { + if (hw->vpu_disable_pending[idx]) { + hw->vpu_disable_pending[idx] = false; + raw_spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_disable_bus_watcher(pmu, MON_VPU, idx); + raw_spin_lock_irq(&pmu->lock); + } + if (hw->monitored_bus[idx] >= 0 && !hw->enabled[idx]) { + int bus = hw->monitored_bus[idx]; + int filter = hw->filter[idx]; + unsigned int g = hw->config_gen[idx]; + + raw_spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON_VPU, idx, bus, filter); + raw_spin_lock_irq(&pmu->lock); + hw->monitor_running = true; + /* Leave it disabled if the watcher was reassigned meanwhile */ + if (hw->config_gen[idx] == g) + WRITE_ONCE(hw->enabled[idx], true); + } + gen[idx] = hw->config_gen[idx]; + valid[idx] = hw->enabled[idx]; + any |= valid[idx]; + } + + if (any) { + int err; + + raw_spin_unlock_irq(&pmu->lock); + err = rpi_axi_pmu_vpu_read_watchers(pmu, regs); + raw_spin_lock_irq(&pmu->lock); + + for (int i = 0; !err && i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + int idx; + u32 new_count; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) != MON_VPU) + continue; + idx = event->hw.idx; + if (idx < 0 || !valid[idx] || hw->config_gen[idx] != gen[idx]) + continue; + counter = config_to_counter(event->attr.config); + new_count = regs[(watcher_offset(idx) + counter_offset(counter) - BW0_CTRL) / 4]; + if (!(event->hw.state & PERF_HES_UPTODATE)) { + event->hw.state |= PERF_HES_UPTODATE; + if (!counter_is_level(counter)) { + local64_set(&event->hw.prev_count, new_count); + continue; + } + } + rpi_axi_pmu_event_update(event, new_count); + } + } + + if (hw->num_monitored == 0 && hw->monitor_running) { + raw_spin_unlock_irq(&pmu->lock); + set_monitor_control(pmu, MON_VPU, GEN_CTL_RESET_BIT); + raw_spin_lock_irq(&pmu->lock); + hw->monitor_running = false; + } + + raw_spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit counter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in softirq / atomic context. + * Holding pmu->lock, reads System monitor counters (MON_SYSTEM) via fast MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *timer) +{ + struct rpi_axi_pmu *pmu = container_of(timer, struct rpi_axi_pmu, hrtimer); + unsigned long flags; + + raw_spin_lock_irqsave(&pmu->lock, flags); + if (pmu->active_events == 0) { + raw_spin_unlock_irqrestore(&pmu->lock, flags); + return HRTIMER_NORESTART; + } + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event = pmu->events[i]; + enum counter counter; + enum monitor mon; + u32 new_count; + + if (!event || (event->hw.state & PERF_HES_STOPPED)) + continue; + mon = config_to_monitor(event->attr.config); + if (pmu->monitor[mon].use_mailbox_interface) + continue; + counter = config_to_counter(event->attr.config); + new_count = rpi_axi_pmu_read_counter(pmu, mon, + event->hw.idx, counter); + rpi_axi_pmu_event_update(event, new_count); + } + + if (pmu->active_vpu_events > 0 && pmu->monitor[MON_VPU].use_mailbox_interface) + schedule_work(&pmu->vpu_work); + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + raw_spin_unlock_irqrestore(&pmu->lock, flags); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + enum counter counter = config_to_counter(event->attr.config); + unsigned long spinflags; + + raw_spin_lock_irqsave(&pmu->lock, spinflags); + if (event->hw.idx < 0) { + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + event->hw.state = 0; + if (!pmu->monitor[mon].use_mailbox_interface) { + int bus = pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter = pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + WRITE_ONCE(pmu->monitor[mon].hw_events.enabled[event->hw.idx], true); + pmu->monitor[mon].hw_events.monitor_running = true; + if (!counter_is_level(counter)) + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |= PERF_HES_UPTODATE; + } else { + schedule_work(&pmu->vpu_work); + } + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events = &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot = -1; + + raw_spin_lock_irqsave(&pmu->lock, spinflags); + idx = rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot = i; + pmu->events[i] = event; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] == 0) { + hw_events->monitored_bus[idx] = -1; + hw_events->filter[idx] = 0; + hw_events->config_gen[idx]++; + hw_events->num_monitored--; + if (pmu->monitor[mon].use_mailbox_interface) { + hw_events->vpu_disable_pending[idx] = true; + schedule_work(&pmu->vpu_work); + } else { + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + if (hw_events->num_monitored == 0) { + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + hw_events->monitor_running = false; + } + } + } + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon == MON_VPU) + pmu->active_vpu_events++; + if (pmu->active_events == 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + event->hw.idx = idx; + event->hw.state = PERF_HES_STOPPED; + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + if (flags & PERF_EF_UPDATE) + rpi_axi_pmu_update(event, false); + raw_spin_lock_irqsave(&pmu->lock, spinflags); + if ((flags & PERF_EF_UPDATE) && !pmu->monitor[mon].use_mailbox_interface) + event->hw.state |= PERF_HES_UPTODATE; + event->hw.state |= PERF_HES_STOPPED; + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu = pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon = config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx = event->hw.idx; + + if (idx < 0) + return; + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + raw_spin_lock_irqsave(&pmu->lock, spinflags); + for (int i = 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] == event) { + pmu->events[i] = NULL; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >= 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] == 0) { + pmu->monitor[mon].hw_events.monitored_bus[idx] = -1; + pmu->monitor[mon].hw_events.filter[idx] = 0; + pmu->monitor[mon].hw_events.config_gen[idx]++; + pmu->monitor[mon].hw_events.num_monitored--; + if (!pmu->monitor[mon].use_mailbox_interface) { + WRITE_ONCE(pmu->monitor[mon].hw_events.enabled[idx], false); + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + if (pmu->monitor[mon].hw_events.num_monitored == 0) { + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + pmu->monitor[mon].hw_events.monitor_running = false; + } + } else { + WRITE_ONCE(pmu->monitor[mon].hw_events.enabled[idx], false); + pmu->monitor[mon].hw_events.vpu_disable_pending[idx] = true; + schedule_work(&pmu->vpu_work); + } + } + } + + event->hw.idx = -1; + pmu->active_events--; + if (mon == MON_VPU) + pmu->active_vpu_events--; + raw_spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_online_cpu() - CPU hotplug callback when a CPU comes online + * @cpu: CPU core number coming online + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_online_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + + if (pmu->cpu == -1) + pmu->cpu = cpu; + return 0; +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designated CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct rpi_axi_pmu *pmu = hlist_entry_safe(node, struct rpi_axi_pmu, cpuhp_node); + unsigned int target; + + if (cpu != pmu->cpu) + return 0; + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) { + pmu->cpu = -1; + return 0; + } + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu = target; + return 0; +} + +/* + * Some firmware accepts GET/SET_PERIPH_REG for the VPU monitor address without + * reaching the hardware, so check that a written enable bit reads back. + */ +static bool rpi_axi_pmu_vpu_accessible(struct rpi_axi_pmu *pmu) +{ + u32 addr = pmu->monitor[MON_VPU].mailbox + GEN_CTRL; + __le32 tmp[3] = { + cpu_to_le32(addr), + cpu_to_le32(1), + cpu_to_le32(GEN_CTL_ENABLE_BIT), + }; + bool ok; + + if (rpi_firmware_property(pmu->firmware, RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)) || le32_to_cpu(tmp[1]) != 1) + return false; + + tmp[0] = cpu_to_le32(addr); + tmp[1] = cpu_to_le32(1); + tmp[2] = cpu_to_le32(0); + ok = !rpi_firmware_property(pmu->firmware, RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)) && + le32_to_cpu(tmp[1]) == 1 && + (le32_to_cpu(tmp[2]) & GEN_CTL_ENABLE_BIT); + + tmp[0] = cpu_to_le32(addr); + tmp[1] = cpu_to_le32(1); + tmp[2] = cpu_to_le32(0); + rpi_firmware_property(pmu->firmware, RPI_FIRMWARE_SET_PERIPH_REG, tmp, sizeof(tmp)); + + return ok; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *fw_node; + int ret; + + raw_spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + pmu->chip = (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); + pmu->pmu = (struct pmu) { + .module = THIS_MODULE, + .task_ctx_nr = perf_invalid_context, + .event_init = rpi_axi_pmu_event_init, + .add = rpi_axi_pmu_add, + .del = rpi_axi_pmu_del, + .start = rpi_axi_pmu_start, + .stop = rpi_axi_pmu_stop, + .read = rpi_axi_pmu_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, + }; + pmu->pdev = pdev; + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, + HRTIMER_MODE_REL_SOFT); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + pmu->monitor[MON_SYSTEM].use_mailbox_interface = false; + pmu->monitor[MON_VPU].use_mailbox_interface = false; + if (IS_ENABLED(CONFIG_RASPBERRYPI_FIRMWARE)) { + fw_node = rpi_firmware_find_node(); + if (!fw_node) { + dev_err(dev, "Missing firmware node\n"); + return -ENOENT; + } + pmu->firmware = devm_rpi_firmware_get(dev, fw_node); + of_node_put(fw_node); + if (!pmu->firmware) + return -EPROBE_DEFER; + pmu->monitor[MON_VPU].use_mailbox_interface = true; + } + + for (int i = 0; i < MON_MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + if (pmu->monitor[i].use_mailbox_interface) { + int addr_cells = of_n_addr_cells(dev->of_node); + int size_cells = of_n_size_cells(dev->of_node); + int index = i * (addr_cells + size_cells) + (addr_cells - 1); + + if (of_property_read_u32_index(dev->of_node, "reg", index, + &pmu->monitor[i].mailbox)) { + dev_err(dev, "Error reading mailbox resource %d\n", i); + ret = -EINVAL; + return ret; + } + } else if (i == MON_SYSTEM) { + struct resource *resource = platform_get_resource(pdev, IORESOURCE_MEM, i); + + if (!resource) + continue; + pmu->monitor[i].base_address = devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret = PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + return ret; + } + } + } + + if (pmu->monitor[MON_VPU].use_mailbox_interface && + !rpi_axi_pmu_vpu_accessible(pmu)) { + dev_warn(dev, "VPU monitor not accessible through firmware\n"); + pmu->monitor[MON_VPU].use_mailbox_interface = false; + pmu->monitor[MON_VPU].base_address = NULL; + } + + /* + * VPU counters are only read every timer interval, and cannot be read when + * an event is stopped, so a short multiplexing slice loses most of a VPU count. + */ + if (pmu->monitor[MON_VPU].use_mailbox_interface) + pmu->pmu.hrtimer_interval_ms = 10 * ktime_to_ms(RPI_AXI_PMU_TIMER_INTERVAL); + + pmu->cpu = -1; + ret = cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_teardown; + } + + if (pmu->chip == CHIP_BCM2711) + pmu->pmu.attr_groups = (const struct attribute_group **)rpi_axi_pmu_bcm2711_attr_groups; + else + pmu->pmu.attr_groups = (const struct attribute_group **)rpi_axi_pmu_bcm2835_attr_groups; + ret = perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=*/-1); + if (ret) { + cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + dev_err(dev, "PMU register failed %d\n", ret); + goto err_teardown; + } + + return 0; +err_teardown: + hrtimer_cancel(&pmu->hrtimer); + flush_work(&pmu->vpu_work); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_node); + /* + * perf_pmu_unregister implicitly detaches all active events, which triggers + * rpi_axi_pmu_del(). For VPU events, this conservatively schedules vpu_work + * to disable the hardware via Mailbox IPC in process context. + */ + perf_pmu_unregister(&pmu->pmu); + hrtimer_cancel(&pmu->hrtimer); + /* + * flush_work MUST be used instead of cancel_work_sync. If we cancel it, + * the deferred hardware disable commands emitted by perf_pmu_unregister + * are silently dropped, permanently orphaning and leaving the VideoCore VPU + * monitor running indefinitely. + * + * Note: Because perf_pmu_unregister and hrtimer_cancel have both completed, + * the work queue is permanently sealed. No new work can be scheduled, guaranteeing + * flush_work cannot race and is completely safe from Use-After-Free during unload. + */ + flush_work(&pmu->vpu_work); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu = platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] = { + { + .compatible = "brcm,bcm2835-axiperf", + .data = (void *)CHIP_BCM2835, + }, + { + .compatible = "brcm,bcm2711-axiperf", + .data = (void *)CHIP_BCM2711, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver = { + .probe = rpi_axi_pmu_probe, + .remove = rpi_axi_pmu_remove, + .driver = { + .name = PMU_NAME, + .of_match_table = rpi_axi_pmu_match, + .suppress_bind_attrs = true, + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:online", + rpi_axi_pmu_online_cpu, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + rpi_axi_pmu_cpuhp_state = ret; + ret = platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); +MODULE_AUTHOR("Ian Rogers "); +MODULE_DESCRIPTION("Broadcom Raspberry Pi AXI Performance Monitor driver"); +MODULE_LICENSE("GPL"); -- 2.56.0.385.gac671c819a-goog