[PATCH 0/5] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters

[PATCH 0/5] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters

[Vladimir Kondratiev]

The Coherence Manager 3 (CM3) of the MIPS I6500 multiprocessing system
is the point where the requests of all cores, I/O coherence units and
the hardware prefetcher are serialized before reaching the shared L2
cache and the memory interface. It carries a small set of performance
counters which observe this traffic: two programmable 32-bit event
counters and a 32-bit cycle counter, programmed through the GCR global
debug block (GCR_DB_PC_*).

The counters can count requests serialized by the CM, requests received
by an IOCU, requests issued on the memory interface, AXI handshake
cycles on the memory and IOCU buses, and PDTrace statistics. Each event
is refined by a 64-bit qualifier made of groups of bits (originating
port, thread, command, cacheability, L1/L2 line state, eviction, bank,
scheduler, ...): a request is counted when it matches a set bit in
every group, so the same event can be narrowed down to, for example,
L2 misses caused by coherent reads from core 1.

This is the only visibility into the L2 and the memory interface that
the I6500 offers; the per-core counters stop at the L1. It is the
natural place to attribute memory bandwidth to cores and IOCUs, or to
measure the L2 hit rate of a workload, on SoCs such as the Mobileye
EyeQ6H.

This series adds an uncore perf driver for these counters.

Design notes
------------

The driver binds to the existing coherency-manager device tree node
("mti,mips-cm" / "mobileye,eyeq6-cm") and reuses the GCR mapping
already set up by mips_cm_probe(), so no new reg property is needed:
the GCR base is discovered from CP0 CMGCRBase as for everything else in
the CM. The register definitions go into asm/mips-cm.h next to the
other GCRs, following the existing accessor scheme.

Events are described by two fields: 'event' (config:0-8) is the number
written to GCR_DB_PC_EVENT, with 0x100 as a pseudo event selecting the
cycle counter, and 'qual' (config1:0-63) is the raw qualifier. A
qualifier of 0 picks a per-event default counting every occurrence;
this matters for the I/O and AXI events, whose qualifiers contain
encoded values (IOCU number, AXI channel) rather than masks, so "all
ones" is not a sensible default there. Named events cover the common
combinations and, since they are plain event=/qual= strings, serve as
templates for custom qualifiers.

A multi-cluster system has one CM3, and thus one set of counters, per
cluster. The number of clusters comes from GCR_CONFIG and one PMU is
registered per cluster, named mips_cm3_pmu_<cluster>. A cluster need
not have CPUs, online or at all, for its CM to be busy: its L2 and
memory interface may serve the I/O devices behind its IOCUs while its
cores are absent or powered down. The counters of a cluster are thus
accessed through the GCR redirect block (GCR_Cx_OTHER with BLOCK set
to the debug block) from whichever CPU the PMU runs on, which may be in
any cluster. The one thing required is that the CM of the counted
cluster is powered, which the driver checks in the CPC (the CM "core"
STAT_CONF sequencer state) when an event is opened; the counters of a
cluster that is powered down at probe time are initialized on first
use, and events on it fail with -ENODEV until then. The counters are
shared by the whole cluster: events are system-wide only, pinned to one
CPU which follows CPU hotplug, and sampling, task events and the
exclude_* attributes are rejected.

Overflow is where the SoC integration leaks in. The CM3 has a
so_cm_perf_cnt_int output, but whether it is wired to a GIC input is
the integrator's choice and is not discoverable. The binding therefore
gains an optional 'interrupts' property: when present the driver uses
it to extend the 32-bit counters, otherwise it polls them every 250 ms,
which is comfortably below the time the fastest event needs to wrap
at the CM3 clock rates. Both paths yield 64-bit counts. No DTS change
is included because the EyeQ6H routing of that signal is not documented
on my side; the driver works in polling mode without it.

The interrupt can only serve the cluster the CPUs live in, and only
when they all live in that one cluster. Each cluster's CM drives its
own interrupt output into its own GIC, and the GIC driver routes a
shared interrupt to the cluster of its affinity CPU, so a single
'interrupts' entry cannot serve several clusters at once. The PMUs of
the other clusters, and all PMUs when CPUs are spread over clusters,
are polled.

The driver can be built as a module. That requires exporting
mips_gcr_base, mips_cpc_base, mips_cm_{lock,unlock}_other() and, less
obviously, mips_cm_is64, since the CPS accessor macros consult it on
every 64-bit GCR access. All are exported _GPL.

Patches
-------

Patch 1 adds the GCR_DB_PC_* register and field definitions, with
local and redirect accessors.

Patch 2 exports the CM and CPC symbols needed by a modular user of the
GCR accessors.

Patch 3 extends the mti,mips-cm binding with the optional performance
counter interrupt.

Patch 4 is the driver itself together with its admin-guide document,
which describes the perf interface and transcribes the event numbers
and qualifier layouts so that users do not need the TRM to build a
qualifier.

Patch 5 enables the driver (as a module) in eyeq6_defconfig.

Testing
-------

Built with eyeq6_defconfig with the driver both built-in and as a
module; every patch of the series builds on its own, and the binding
and the document pass their respective checkers.

Run on the EyeQ6H board with 2 clusters, loaded as module in polling mode.

[   66.128375] mips_cm3_pmu coherency-manager: mips_cm3_pmu_0: 2 counters + cycles, polled
[   66.129617] mips_cm3_pmu coherency-manager: mips_cm3_pmu_1: 2 counters + cycles, polled

dev / # perf stat -a -e mips_cm3_pmu_0/cm_requests_l2_hit/,mips_cm3_pmu_0/cm_requests_l2_miss/,mips_cm3_p
mu_1/cm_requests_l2_hit/,mips_cm3_pmu_1/cm_requests_l2_miss/,mips_cm3_pmu_0/cycles/,mips_cm3_pmu_1/cycles
/ sleep 1

 Performance counter stats for 'system wide':

            505821      mips_cm3_pmu_0/cm_requests_l2_hit/                                   
             49083      mips_cm3_pmu_0/cm_requests_l2_miss/                                   
            184893      mips_cm3_pmu_1/cm_requests_l2_hit/                                   
             37760      mips_cm3_pmu_1/cm_requests_l2_miss/                                   
        1807514589      mips_cm3_pmu_0/cycles/                                      
        1807491360      mips_cm3_pmu_1/cycles/                                      

       1.005857149 seconds time elapsed

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
Vladimir Kondratiev (5):
      MIPS: CM: add CM3 performance counter GCR definitions
      MIPS: CM: export the symbols needed to access the GCRs from a module
      dt-bindings: mips: mti,mips-cm: add performance counter interrupt
      perf: add MIPS Coherence Manager 3 (CM3) PMU driver
      MIPS: eyeq6_defconfig: enable the CM3 PMU

 Documentation/admin-guide/perf/index.rst           |   1 +
 Documentation/admin-guide/perf/mips_cm3_pmu.rst    | 337 +++++++
 .../devicetree/bindings/mips/mti,mips-cm.yaml      |   6 +
 arch/mips/configs/eyeq6_defconfig                  |   2 +
 arch/mips/include/asm/mips-cm.h                    |  51 +-
 arch/mips/kernel/mips-cm.c                         |   5 +
 arch/mips/kernel/mips-cpc.c                        |   2 +
 drivers/perf/Kconfig                               |  12 +
 drivers/perf/Makefile                              |   1 +
 drivers/perf/mips_cm3_pmu.c                        | 988 +++++++++++++++++++++
 10 files changed, 1404 insertions(+), 1 deletion(-)
---
base-commit: c03c1a14507dfaf8b856c2608f9ac216021fc222
change-id: 20261101-i6500-cm3-pmu-4a11f399b430

Best regards,
--  
Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>

[PATCH 1/5] MIPS: CM: add CM3 performance counter GCR definitions

[Vladimir Kondratiev]

The CM3 implements two programmable 32-bit performance counters and a
32-bit cycle counter, controlled through registers in the GCR global
debug block (GCR_DB_PC_*, offset 0x6100..0x61a8 from GCR_BASE).

Add GCR_DB_ACCESSOR_{RO,RW} helpers for the debug block and define the
control, overflow status, event select, cycle counter, qualifier and
counter registers along with their fields, as described in the I6500
register reference (MD01189).

Each accessor comes in two flavours, as the CPC ones do: db_<reg> for
the local cluster and redir_db_<reg> for the debug block of another
cluster through the redirect block. GCR_Cx_OTHER.BLOCK value 2 selects
the debug block in the GCR, where the existing _BLOCK_USER name only
describes its GIC meaning; add CM_GCR_Cx_OTHER_BLOCK_DEBUG for it.

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
 arch/mips/include/asm/mips-cm.h | 51 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h
index 407f253bb4a1..390d06a8f1d1 100644
--- a/arch/mips/include/asm/mips-cm.h
+++ b/arch/mips/include/asm/mips-cm.h
@@ -161,6 +161,20 @@ static inline bool mips_cm_has_l2sync(void)
 	CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_CLCB_OFS + off, cl_##name)	\
 	CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_COCB_OFS + off, co_##name)
 
+/*
+ * Global debug block registers. The redir_db_* accessors reach the debug
+ * block of another cluster through the redirect block, which must have been
+ * pointed at it with mips_cm_lock_other(cluster, 0, 0,
+ * CM_GCR_Cx_OTHER_BLOCK_DEBUG).
+ */
+#define GCR_DB_ACCESSOR_RO(sz, off, name)				\
+	CPS_ACCESSOR_RO(gcr, sz, MIPS_CM_GDB_OFS + off, db_##name)	\
+	CPS_ACCESSOR_RO(gcr, sz, MIPS_CM_COCB_OFS + off, redir_db_##name)
+
+#define GCR_DB_ACCESSOR_RW(sz, off, name)				\
+	CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_GDB_OFS + off, db_##name)	\
+	CPS_ACCESSOR_RW(gcr, sz, MIPS_CM_COCB_OFS + off, redir_db_##name)
+
 /* GCR_CONFIG - Information about the system */
 GCR_ACCESSOR_RO(64, 0x000, config)
 #define CM_GCR_CONFIG_CLUSTER_COH_CAPABLE	BIT_ULL(43)
@@ -357,7 +371,8 @@ GCR_CX_ACCESSOR_RW(32, 0x018, other)
 #define CM_GCR_Cx_OTHER_BLOCK			GENMASK(25, 24)	/* CM >= 3.5 */
 #define  CM_GCR_Cx_OTHER_BLOCK_LOCAL		0
 #define  CM_GCR_Cx_OTHER_BLOCK_GLOBAL		1
-#define  CM_GCR_Cx_OTHER_BLOCK_USER		2
+#define  CM_GCR_Cx_OTHER_BLOCK_USER		2	/* GIC */
+#define  CM_GCR_Cx_OTHER_BLOCK_DEBUG		2	/* GCR */
 #define  CM_GCR_Cx_OTHER_BLOCK_GLOBAL_HIGH	3
 #define CM_GCR_Cx_OTHER_CLUSTER			GENMASK(21, 16)	/* CM >= 3.5 */
 #define CM3_GCR_Cx_OTHER_CORE			GENMASK(13, 8)	/* CM >= 3 */
@@ -384,6 +399,40 @@ GCR_CX_ACCESSOR_RW(32, 0x030, reset_ext_base)
 #define CM_GCR_Cx_RESET_EXT_BASE_BEVEXCPA	GENMASK(7, 1)
 #define CM_GCR_Cx_RESET_EXT_BASE_PRESENT	BIT(0)
 
+/* GCR_DB_PC_CTL - Control of the CM3 performance counters */
+GCR_DB_ACCESSOR_RW(32, 0x100, pc_ctl)
+#define CM_GCR_DB_PC_CTL_PERF_INT_EN		BIT(30)
+#define CM_GCR_DB_PC_CTL_PERF_OVF_STOP		BIT(29)
+#define CM_GCR_DB_PC_CTL_P1_RESET		BIT(9)
+#define CM_GCR_DB_PC_CTL_P1_COUNTON		BIT(8)
+#define CM_GCR_DB_PC_CTL_P0_RESET		BIT(7)
+#define CM_GCR_DB_PC_CTL_P0_COUNTON		BIT(6)
+#define CM_GCR_DB_PC_CTL_CYCL_CNT_RESET		BIT(5)
+#define CM_GCR_DB_PC_CTL_CYCL_CNT_COUNTON	BIT(4)
+#define CM_GCR_DB_PC_CTL_PERF_NUM_CNT		GENMASK(3, 0)
+
+/* GCR_DB_PC_OV - CM3 performance counter overflow status (write 1 to clear) */
+GCR_DB_ACCESSOR_RW(32, 0x120, pc_ov)
+#define CM_GCR_DB_PC_OV_P1_OF			BIT(2)
+#define CM_GCR_DB_PC_OV_P0_OF			BIT(1)
+#define CM_GCR_DB_PC_OV_CYCL_CNT_OF		BIT(0)
+
+/* GCR_DB_PC_EVENT - CM3 performance counter event selection */
+GCR_DB_ACCESSOR_RW(32, 0x130, pc_event)
+#define CM_GCR_DB_PC_EVENT_P1_EVENT		GENMASK(15, 8)
+#define CM_GCR_DB_PC_EVENT_P0_EVENT		GENMASK(7, 0)
+
+/* GCR_DB_PC_CYCL - CM3 performance cycle counter */
+GCR_DB_ACCESSOR_RW(32, 0x180, pc_cycl)
+
+/* GCR_DB_PC_QUALn - CM3 performance counter n event qualifier */
+GCR_DB_ACCESSOR_RW(64, 0x190, pc_qual0)
+GCR_DB_ACCESSOR_RW(64, 0x1a0, pc_qual1)
+
+/* GCR_DB_PC_CNTn - CM3 performance counter n value */
+GCR_DB_ACCESSOR_RW(32, 0x198, pc_cnt0)
+GCR_DB_ACCESSOR_RW(32, 0x1a8, pc_cnt1)
+
 /**
  * mips_cm_l2sync - perform an L2-only sync operation
  *

-- 
2.43.0

[PATCH 2/5] MIPS: CM: export the symbols needed to access the GCRs from a module

[Vladimir Kondratiev]

Allow modules to use the GCR and CPC accessors from asm/mips-cm.h and
asm/mips-cpc.h:

- mips_gcr_base and mips_cpc_base are the mapped register blocks;
- the CPS_ACCESSOR macros consult mips_cm_is64 on every 64-bit register
  access;
- mips_cm_lock_other() / mips_cm_unlock_other() are the only way to
  reach the registers of another cluster through the redirect block.

This is a prerequisite for building the CM3 PMU driver as a module.

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
 arch/mips/kernel/mips-cm.c  | 5 +++++
 arch/mips/kernel/mips-cpc.c | 2 ++
 2 files changed, 7 insertions(+)

diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c
index 7c9c5dc38823..afe0664e4dcf 100644
--- a/arch/mips/kernel/mips-cm.c
+++ b/arch/mips/kernel/mips-cm.c
@@ -5,6 +5,7 @@
  */
 
 #include <linux/errno.h>
+#include <linux/export.h>
 #include <linux/of.h>
 #include <linux/percpu.h>
 #include <linux/spinlock.h>
@@ -14,8 +15,10 @@
 #include <asm/mipsregs.h>
 
 void __iomem *mips_gcr_base;
+EXPORT_SYMBOL_GPL(mips_gcr_base);
 void __iomem *mips_cm_l2sync_base;
 int mips_cm_is64;
+EXPORT_SYMBOL_GPL(mips_cm_is64);
 bool mips_cm_is_l2_hci_broken;
 
 static char *cm2_tr[8] = {
@@ -373,6 +376,7 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core,
 	 */
 	mb();
 }
+EXPORT_SYMBOL_GPL(mips_cm_lock_other);
 
 void mips_cm_unlock_other(void)
 {
@@ -389,6 +393,7 @@ void mips_cm_unlock_other(void)
 
 	preempt_enable();
 }
+EXPORT_SYMBOL_GPL(mips_cm_unlock_other);
 
 void mips_cm_error_report(void)
 {
diff --git a/arch/mips/kernel/mips-cpc.c b/arch/mips/kernel/mips-cpc.c
index 3e386f7e1545..4a8bd56bc2e8 100644
--- a/arch/mips/kernel/mips-cpc.c
+++ b/arch/mips/kernel/mips-cpc.c
@@ -6,6 +6,7 @@
 
 #include <linux/bitfield.h>
 #include <linux/errno.h>
+#include <linux/export.h>
 #include <linux/percpu.h>
 #include <linux/of.h>
 #include <linux/of_address.h>
@@ -14,6 +15,7 @@
 #include <asm/mips-cps.h>
 
 void __iomem *mips_cpc_base;
+EXPORT_SYMBOL_GPL(mips_cpc_base);
 
 static DEFINE_PER_CPU_ALIGNED(spinlock_t, cpc_core_lock);
 

-- 
2.43.0

[PATCH 3/5] dt-bindings: mips: mti,mips-cm: add performance counter interrupt

[Vladimir Kondratiev]

The CM3 asserts so_cm_perf_cnt_int when one of its performance counters
overflows. Whether that output is connected to an interrupt controller
input is up to the SoC integrator, so describe it as an optional
'interrupts' property of the coherency-manager node. When absent, the
counters have to be polled by software.

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
 Documentation/devicetree/bindings/mips/mti,mips-cm.yaml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
index d129d6382847..ca7a6717ecdf 100644
--- a/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
+++ b/Documentation/devicetree/bindings/mips/mti,mips-cm.yaml
@@ -38,6 +38,12 @@ properties:
 
     maxItems: 1
 
+  interrupts:
+    description:
+      Performance counter overflow interrupt of the CM (so_cm_perf_cnt_int).
+      When absent, the performance counters are polled by software.
+    maxItems: 1
+
 required:
   - compatible
 

-- 
2.43.0

[PATCH 4/5] perf: add MIPS Coherence Manager 3 (CM3) PMU driver

[Vladimir Kondratiev]

The CM3 of the MIPS I6500 multiprocessing system serializes the requests
of all cores, IOCUs and the hardware prefetcher before they reach the
shared L2 and the memory interface. It provides two programmable 32-bit
event counters and a 32-bit cycle counter to observe this traffic:
coherence manager requests, I/O and memory interface requests, AXI bus
utilization and PDTrace statistics. Each event is refined by a 64-bit
qualifier split into groups of bits (port, thread, command, CCA, L1/L2
state, ...), where a request is counted when it matches a set bit in
every group.

Expose the counters as uncore PMUs bound to the coherency-manager DT
node, using the GCR block already mapped by mips_cm_probe(). Events are
described by 'event' (config:0-8, 0x100 selecting the cycle counter)
and 'qual' (config1:0-63); a qualifier of 0 picks a per-event default
that counts every occurrence. Named events are provided for the common
combinations and double as templates for custom qualifiers.

Every cluster has its own CM3 and its own counters, so one PMU named
mips_cm3_pmu_<cluster> is registered per cluster found in GCR_CONFIG.
A cluster does not need CPUs, online or at all, for its L2 and memory
interface to be busy, so the counters of other clusters are accessed
through the GCR redirect block from whichever CPU runs the PMU. This
only requires the CM of that cluster to be powered, which is checked
in the CPC (CM core STAT_CONF) when an event is opened: the counters
of a powered down cluster are initialized lazily on first use and
events on it fail with -ENODEV meanwhile. The counters are shared by
the whole cluster: events are system-wide only, pinned to a single CPU
which follows CPU hotplug; sampling, task events and the exclude_*
attributes are rejected.

Counter overflow is handled through the CM interrupt when the DT node
provides one, and by periodic polling otherwise, giving 64-bit counts
in both cases. Each cluster's CM drives its own interrupt output into
its own GIC, and a GIC shared interrupt is only routed to the cluster
of its affinity CPU, so the interrupt serves the cluster the CPUs live
in, and only when they all live in that one cluster; everything else
is polled.

Document the perf interface, the event numbers and the qualifier layouts
in Documentation/admin-guide/perf/mips_cm3_pmu.rst.

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
 Documentation/admin-guide/perf/index.rst        |   1 +
 Documentation/admin-guide/perf/mips_cm3_pmu.rst | 337 ++++++++
 drivers/perf/Kconfig                            |  12 +
 drivers/perf/Makefile                           |   1 +
 drivers/perf/mips_cm3_pmu.c                     | 988 ++++++++++++++++++++++++
 5 files changed, 1339 insertions(+)

diff --git a/Documentation/admin-guide/perf/index.rst b/Documentation/admin-guide/perf/index.rst
index aa12708ddb96..97620aa20465 100644
--- a/Documentation/admin-guide/perf/index.rst
+++ b/Documentation/admin-guide/perf/index.rst
@@ -14,6 +14,7 @@ Performance monitor support
    qcom_l2_pmu
    qcom_l3_pmu
    starfive_starlink_pmu
+   mips_cm3_pmu
    mrvl-odyssey-ddr-pmu
    mrvl-odyssey-tad-pmu
    arm-ccn
diff --git a/Documentation/admin-guide/perf/mips_cm3_pmu.rst b/Documentation/admin-guide/perf/mips_cm3_pmu.rst
new file mode 100644
index 000000000000..33cba0aca494
--- /dev/null
+++ b/Documentation/admin-guide/perf/mips_cm3_pmu.rst
@@ -0,0 +1,337 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================================================================
+MIPS Coherence Manager 3 (CM3) Performance Monitoring Unit (PMU)
+=================================================================
+
+The Coherence Manager 3 (CM3) of the MIPS I6500 multiprocessing system is
+the point where the requests of all cores, I/O coherence units (IOCU) and
+the hardware prefetcher are serialized before reaching the shared L2 cache
+and the memory interface. The CM3 implements a small set of performance
+counters which observe this traffic:
+
+- two programmable 32-bit event counters, and
+- one 32-bit cycle counter clocked at the CM3 clock.
+
+The counters live in the global debug block of the CM Global Configuration
+Registers (``GCR_DB_PC_*``), are shared by all CPUs of the cluster, and count
+regardless of which CPU, privilege level or task generated the traffic. They
+are exposed by the ``mips_cm3_pmu`` driver (``CONFIG_MIPS_CM3_PMU``, built-in
+or as the ``mips_cm3_pmu`` module) as uncore PMUs.
+
+Clusters
+========
+
+A multi-cluster system has one CM3, and so one set of counters, per
+cluster. The number of clusters is read from ``GCR_CONFIG`` and the driver
+registers one PMU per cluster, named after the cluster number::
+
+  /sys/bus/event_source/devices/mips_cm3_pmu_0/
+  /sys/bus/event_source/devices/mips_cm3_pmu_1/
+
+A single-cluster system only has ``mips_cm3_pmu_0``. Each PMU counts the
+traffic of its own cluster: a request from a core in cluster 1 to the memory
+interface of cluster 1 is seen by ``mips_cm3_pmu_1`` only.
+
+A cluster does not need CPUs for its CM to be busy: its L2 cache and memory
+interface may serve I/O devices behind its IOCUs while its cores are absent,
+offline or powered down. The counters of any cluster are therefore reached
+through the GCR redirect block from whichever CPU runs the PMU, which can be
+in any cluster. What is needed is that the CM of the counted cluster is
+powered: the driver checks the CM power sequencer state in the Cluster Power
+Controller (``CPC_STAT_CONF`` of the CM "core") when an event is opened and
+fails with ``-ENODEV`` if the cluster is powered down. The PMU of the
+cluster the CPUs run in is always usable.
+
+Because the counters are shared, events are system wide only. Sampling
+(``perf record``), attaching to a task, and the ``exclude_*`` attributes are
+not supported; ``perf stat -a`` is the intended interface. The ``cpumask``
+attribute names the single CPU on which the counters are programmed and
+read; it migrates to another online CPU when that one goes offline::
+
+  $ cat /sys/bus/event_source/devices/mips_cm3_pmu_1/cpumask
+  0
+
+Counter overflow
+================
+
+The CM3 raises its ``so_cm_perf_cnt_int`` output when a counter reaches
+0xFFFFFFFF. Whether that output reaches an interrupt controller input is a
+decision of the SoC integrator. If the ``coherency-manager`` device tree node
+carries an ``interrupts`` property, the driver uses it to extend the counters
+to 64 bits. Otherwise the driver polls the counters every 250 ms, which is
+well below the time the fastest event needs to wrap a 32-bit counter. Either
+way, the values reported by perf are 64-bit.
+
+The interrupt can only serve the cluster the CPUs live in, and only when
+they all live in that one cluster. Every cluster's CM drives its own
+interrupt output into its own GIC, and a GIC shared interrupt is routed to
+the cluster of its affinity CPU, so a single ``interrupts`` entry cannot
+serve several clusters at once. The PMUs of all other clusters, and all PMUs
+when CPUs are spread over several clusters, are polled.
+
+The mode chosen is printed at probe time, or the fact that the cluster was
+powered down when the driver probed; the counters are then initialized when
+the first event is opened::
+
+  mips_cm3_pmu coherency-manager: mips_cm3_pmu_0: 2 counters + cycles, polled
+  mips_cm3_pmu coherency-manager: mips_cm3_pmu_1: cluster powered down
+
+Event encoding
+==============
+
+An event is described by two fields, documented in the ``format`` directory::
+
+  $ cat /sys/bus/event_source/devices/mips_cm3_pmu_0/format/event
+  config:0-8
+  $ cat /sys/bus/event_source/devices/mips_cm3_pmu_0/format/qual
+  config1:0-63
+
+``event`` selects the event number written to ``GCR_DB_PC_EVENT``. The
+value ``0x100`` is a pseudo event selecting the cycle counter.
+
+``qual`` is the 64-bit qualifier written to ``GCR_DB_PC_QUALn``. It refines
+the event: the qualifier is split into groups of bits, each group covering
+one attribute of a request (originating port, command, cacheability, L2
+state, ...). A request is counted when, in **every** group, the bit
+matching its attribute is set. Setting several bits of a group ORs the
+alternatives; setting all bits of a group makes it a "don't care". A ``qual``
+of 0 (or omitting it) selects a per-event default that counts every
+occurrence of the event.
+
+Event numbers
+-------------
+
+===========  ==========================  ======================================
+Event        Name                        Description
+===========  ==========================  ======================================
+0x01         CM requests                 Requests serialized by the coherence
+                                         manager (qualifier: Table 1)
+0x02         I/O traffic requests        Requests received by an IOCU
+                                         (qualifier: Table 2)
+0x03         Memory interface requests   Requests issued on the memory
+                                         interface (qualifier: Table 3)
+0x07         MEM AXI utilization         Cycles on the memory AXI/ACE bus
+                                         (qualifier: Table 4)
+0x08 - 0x0f  IOCU0-7 AXI utilization     Cycles on the AXI bus of IOCU
+                                         (event - 0x08) (qualifier: Table 4)
+0x11         PDTrace dropped messages    Messages dropped by CM PDTrace
+                                         (qualifier: Table 5)
+0x12         PDTrace overflow cycles     Cycles spent in CM PDTrace overflow
+0x100        cycles                      CM3 cycle counter (pseudo event)
+===========  ==========================  ======================================
+
+Table 1: Coherence Manager Requests qualifier (event 0x01)
+----------------------------------------------------------
+
+======  ===========  ===================================================
+Bits    Group        Values (one bit each, LSB first)
+======  ===========  ===================================================
+9:0     Port         Input port 0..7, prefetcher (8), intervention (9).
+                     Ports are numbered cores first, then IOCUs.
+13:10   Thread       Thread 0..3
+15:14   Target       Memory, register bus (GCR, GIC, CPC, ...)
+39:16   Command      Legacy read, legacy write, CohReadOwn,
+                     CohReadShare, CohReadDiscard, CohEvict,
+                     CohUpgrade, CohUpgradeSC, CohWriteBack,
+                     CohWriteInvalidate, CohReadDiscardAllocate,
+                     prefetch own, prefetch share, prefetch write
+                     invalidate, GetToOwn, Tag_Err, RegRead,
+                     RegWrite, Sync, L1I cacheop, L1D cacheop,
+                     L2 cacheop, L3 cacheop, other
+42:40   CCA          Uncached, uncached accelerated, other
+44:43   Size         Less than a line, one line
+47:45   L1 state     Invalid, shared, exclusive/modified
+51:48   L2 state     Invalid, shared, exclusive, modified
+53:52   L2 locked    Not locked, locked
+56:54   Eviction     None, L2 and L1 eviction, L2 eviction only
+58:57   Bank         L2 bank 0, bank 1
+60:59   Scheduler    Scheduler 0, scheduler 1
+63:61   Reserved     Must be set
+======  ===========  ===================================================
+
+Default: all bits set.
+
+Table 2: I/O Traffic Requests qualifier (event 0x02)
+----------------------------------------------------
+
+======  ===============  ===============================================
+Bits    Group            Values
+======  ===============  ===============================================
+2:0     IOCU number      Encoded 0..7 (a value, not a mask)
+7:3     Direction/CCA    Uncached read, coherent read with allocate,
+                         coherent read without allocate, uncached
+                         write, coherent write
+15:8    Size             1, 2, 4, 8, 16, 32, 64, 128 bytes per beat
+24:16   Length           1, 2, 3-4, 5-8, 9-16, 17-32, 33-64, 65-128,
+                         129-256 beats
+26:25   Alignment        Aligned, misaligned
+28:27   Prefetch         Not prefetch, prefetch
+29      All device IDs   Set to ignore bits 35:30
+35:30   Device ID        Encoded device ID
+36      All txn IDs      Set to ignore bits 41:37
+41:37   Transaction ID   Encoded transaction ID
+======  ===============  ===============================================
+
+Default: IOCU 0, every other group "don't care" (``0x3fffffffff8``).
+
+Table 3: Memory Interface Requests qualifier (event 0x03)
+---------------------------------------------------------
+
+======  ===============  ===============================================
+Bits    Group            Values
+======  ===============  ===============================================
+1:0     Direction        Read, write
+9:2     Size             1, 2, 4, 8, 16, 32, 64, 128 bytes per beat
+17:10   Length           0..7 beats
+21:18   Cacheability     Uncached, uncached accelerated, cacheable read
+                         discard, cacheable not read discard
+29:22   Source           Input port 0..7
+35:32   Thread           Thread 0..3
+37:36   Code/data        Data, code
+39:38   Prefetch         Not prefetch, prefetch
+40      All guest IDs    Set to ignore bits 45:41
+45:41   Guest ID         Encoded guest ID
+======  ===============  ===============================================
+
+Default: all groups "don't care" (``0x3fffffffffff``).
+
+Table 4: AXI Bus Utilization qualifier (events 0x07 - 0x0f)
+-----------------------------------------------------------
+
+These events count cycles, qualified by the state of the handshake signals
+of one AXI channel.
+
+======  ===============  ===============================================
+Bits    Group            Values
+======  ===============  ===============================================
+0       Not valid        Count cycles where xVALID is deasserted
+1       Valid            Count cycles where xVALID is asserted
+2       Not ready        Count cycles where xREADY is deasserted
+3       Ready            Count cycles where xREADY is asserted
+6:4     Channel          Encoded: 0 AR, 1 AW, 2 W, 3 R, 4 B
+======  ===============  ===============================================
+
+Setting both ``valid`` and ``ready`` counts completed transfers. Default:
+transfers on the AR channel (``0xa``).
+
+Table 5: CM PDTrace Dropped Messages qualifier (event 0x11)
+-----------------------------------------------------------
+
+======  ===============  ===============================================
+Bits    Group            Values
+======  ===============  ===============================================
+5:0     Trace port       ubrh, prsh, tmh0, tmh0_mulp, tmh1, tmh1_mulp
+7:6     Trace type       Request, response
+======  ===============  ===============================================
+
+Default: all bits set.
+
+Named events
+============
+
+The ``events`` directory provides ready-made event/qualifier pairs for the
+most useful combinations::
+
+  $ perf list mips_cm3_pmu_0
+
+  mips_cm3_pmu_0/cm_requests/                          [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_coh_writeback/            [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_l2_hit/                   [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_l2_miss/                  [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_legacy_read/              [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_legacy_write/             [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_mem/                      [Kernel PMU event]
+  mips_cm3_pmu_0/cm_requests_regbus/                   [Kernel PMU event]
+  mips_cm3_pmu_0/cycles/                               [Kernel PMU event]
+  mips_cm3_pmu_0/iocu0_requests/                       [Kernel PMU event]
+  mips_cm3_pmu_0/mem_axi_ar/                           [Kernel PMU event]
+  mips_cm3_pmu_0/mem_axi_aw/                           [Kernel PMU event]
+  mips_cm3_pmu_0/mem_axi_b/                            [Kernel PMU event]
+  mips_cm3_pmu_0/mem_axi_r/                            [Kernel PMU event]
+  mips_cm3_pmu_0/mem_axi_w/                            [Kernel PMU event]
+  mips_cm3_pmu_0/mem_reads/                            [Kernel PMU event]
+  mips_cm3_pmu_0/mem_requests/                         [Kernel PMU event]
+  mips_cm3_pmu_0/mem_writes/                           [Kernel PMU event]
+  mips_cm3_pmu_0/pdtrace_dropped/                      [Kernel PMU event]
+  mips_cm3_pmu_0/pdtrace_overflow_cycles/              [Kernel PMU event]
+
+==============================  =============================================
+Name                            Counts
+==============================  =============================================
+cycles                          CM3 clock cycles
+cm_requests                     Every request serialized by the CM
+cm_requests_mem                 Requests targeting memory
+cm_requests_regbus              Requests targeting the register bus
+cm_requests_legacy_read         Non-coherent reads, including code fetches
+cm_requests_legacy_write        Non-coherent writes
+cm_requests_coh_writeback       Coherent write backs
+cm_requests_l2_hit              Memory requests whose line is present in L2
+cm_requests_l2_miss             Memory requests whose line is not in L2
+iocu0_requests                  Requests received by IOCU 0
+mem_requests                    Requests issued on the memory interface
+mem_reads                       Memory interface reads
+mem_writes                      Memory interface writes
+mem_axi_ar/aw/w/r/b             Completed transfers on the given channel
+                                of the memory AXI bus
+pdtrace_dropped                 Messages dropped by CM PDTrace
+pdtrace_overflow_cycles         Cycles spent in CM PDTrace overflow
+==============================  =============================================
+
+The named events are plain ``event=...,qual=...`` strings, so they serve as
+templates: read one from sysfs and clear the bits of the group you want to
+narrow down.
+
+Examples
+========
+
+Count all CM requests and the share coming from port 1 (core 1), together
+with cycles, for one second::
+
+  $ perf stat -a -e mips_cm3_pmu_0/cycles/ \
+                 -e mips_cm3_pmu_0/cm_requests/ \
+                 -e mips_cm3_pmu_0/event=0x1,qual=0xfffffffffffffc02/ \
+                 sleep 1
+
+The second qualifier is the ``cm_requests`` default with the Port group
+(bits 9:0) reduced to bit 1.
+
+L2 hit rate of memory requests::
+
+  $ perf stat -a -e mips_cm3_pmu_0/cm_requests_l2_hit/ \
+                 -e mips_cm3_pmu_0/cm_requests_l2_miss/ sleep 1
+
+Utilization of the memory read data channel: cycles with a transfer versus
+cycles where the CM waits for data (valid deasserted)::
+
+  $ perf stat -a -e mips_cm3_pmu_0/cycles/ \
+                 -e mips_cm3_pmu_0/mem_axi_r/ \
+                 -e mips_cm3_pmu_0/event=0x7,qual=0x3d/ sleep 1
+
+Requests received by IOCU 1 (IOCU number is an encoded value in bits 2:0)::
+
+  $ perf stat -a -e mips_cm3_pmu_0/event=0x2,qual=0x3fffffffff9/ sleep 1
+
+Memory interface traffic of every cluster of a two-cluster system::
+
+  $ perf stat -a -e mips_cm3_pmu_0/mem_requests/ \
+                 -e mips_cm3_pmu_1/mem_requests/ sleep 1
+
+Only two programmable counters exist; a group requesting more than two
+non-cycle events, or more than one ``cycles`` event, is rejected with
+``-EINVAL``.
+
+Device tree
+===========
+
+The driver binds to the ``coherency-manager`` node (``mti,mips-cm`` or
+``mobileye,eyeq6-cm``) and uses the GCR base address already discovered by
+the kernel from the ``CMGCRBase`` register. An ``interrupts`` property on
+that node, when the SoC routes ``so_cm_perf_cnt_int`` to an interrupt
+controller, enables interrupt-driven overflow handling::
+
+  coherency-manager {
+      compatible = "mobileye,eyeq6-cm";
+      interrupt-parent = <&gic>;
+      interrupts = <GIC_SHARED 42 IRQ_TYPE_LEVEL_HIGH>;
+  };
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..2dbed2fc7b03 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -75,6 +75,18 @@ config ARM_XSCALE_PMU
 	depends on ARM_PMU && CPU_XSCALE
 	def_bool y
 
+config MIPS_CM3_PMU
+	tristate "MIPS Coherence Manager 3 (CM3) PMU"
+	depends on MIPS_CM
+	help
+	  Support for the performance counters of the MIPS Coherence Manager 3
+	  (CM3) found in the I6500 multiprocessing system. The CM3 provides two
+	  programmable event counters and a cycle counter which can count
+	  requests serialized by the coherence manager, I/O and memory interface
+	  traffic, and AXI bus utilization, each refined by a qualifier.
+
+	  If compiled as a module, it will be called mips_cm3_pmu.
+
 config RISCV_PMU
 	depends on RISCV
 	bool "RISC-V PMU framework"
diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile
index eb8a022dad9a..5f0920ba7058 100644
--- a/drivers/perf/Makefile
+++ b/drivers/perf/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_FUJITSU_UNCORE_PMU) += fujitsu_uncore_pmu.o
 obj-$(CONFIG_HISI_PMU) += hisilicon/
 obj-$(CONFIG_QCOM_L2_PMU)	+= qcom_l2_pmu.o
 obj-$(CONFIG_QCOM_L3_PMU) += qcom_l3_pmu.o
+obj-$(CONFIG_MIPS_CM3_PMU) += mips_cm3_pmu.o
 obj-$(CONFIG_RISCV_PMU) += riscv_pmu.o
 obj-$(CONFIG_RISCV_PMU_LEGACY) += riscv_pmu_legacy.o
 obj-$(CONFIG_RISCV_PMU_SBI) += riscv_pmu_sbi.o
diff --git a/drivers/perf/mips_cm3_pmu.c b/drivers/perf/mips_cm3_pmu.c
new file mode 100644
index 000000000000..d4be23e79d6f
--- /dev/null
+++ b/drivers/perf/mips_cm3_pmu.c
@@ -0,0 +1,988 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * MIPS Coherence Manager 3 (CM3) performance counter PMU driver
+ *
+ * The CM3 found in the MIPS I6500 multiprocessing system implements two
+ * programmable 32-bit event counters and a 32-bit cycle counter. They are
+ * controlled through the GCR global debug block (GCR_DB_PC_*) and count
+ * events happening at the coherence manager: serialized requests, I/O and
+ * memory traffic, AXI bus utilization, ...
+ *
+ * perf interface:
+ *
+ *   config  [8:0]  event number (GCR_DB_PC_EVENT.Px_Event), 0x100 selects
+ *                  the cycle counter.
+ *   config1 [63:0] event qualifier (GCR_DB_PC_QUALn). A qualifier is split
+ *                  into groups of bits, each selecting one attribute of the
+ *                  request; an event is counted when it matches at least one
+ *                  set bit of every group. Setting all bits of a group makes
+ *                  it a "don't care". A qualifier of 0 selects a sensible
+ *                  default which counts every occurrence of the event.
+ *
+ * Every cluster has its own CM3 and therefore its own set of counters, and
+ * one PMU instance (mips_cm3_pmu_<cluster>) is registered per cluster. A
+ * cluster does not need CPUs, online or at all, for its L2 and memory
+ * interface to be busy, so the counters of other clusters are reached
+ * through the GCR redirect block from whichever CPU runs the PMU. This
+ * requires the CM of that cluster to be powered, which is checked in the
+ * CPC before an event is opened.
+ *
+ * The counters are shared by the whole cluster, so events are system wide,
+ * pinned to a single CPU, and cannot be sampled.
+ */
+
+#define CM3_PMU_NAME	"mips_cm3_pmu"
+
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+
+#include <asm/cpu-info.h>
+#include <asm/mips-cps.h>
+
+/* Two programmable counters followed by the cycle counter */
+#define CM3_PMU_MAX_PCS			2
+#define CM3_PMU_IDX_CYCLES		CM3_PMU_MAX_PCS
+#define CM3_PMU_NUM_COUNTERS		(CM3_PMU_MAX_PCS + 1)
+#define CM3_PMU_COUNTER_MASK		GENMASK_ULL(31, 0)
+
+/*
+ * Without an overflow interrupt the 32-bit counters are read periodically.
+ * The fastest events count a couple of occurrences per CM clock cycle, so
+ * even at 2GHz a counter takes more than a second to wrap.
+ */
+#define CM3_PMU_POLL_PERIOD_NS		(250 * NSEC_PER_MSEC)
+
+/* Event numbers (GCR_DB_PC_EVENT.Px_Event) */
+#define CM3_PMU_EV_NONE			0x00
+#define CM3_PMU_EV_CM_REQ		0x01	/* Coherence Manager requests */
+#define CM3_PMU_EV_IO_REQ		0x02	/* I/O traffic requests (IOCU) */
+#define CM3_PMU_EV_MEM_REQ		0x03	/* Memory interface requests */
+#define CM3_PMU_EV_MEM_AXI		0x07	/* MEM AXI bus utilization */
+#define CM3_PMU_EV_IOCU_AXI(n)		(0x08 + (n)) /* IOCUn AXI bus utilization */
+#define CM3_PMU_EV_IOCU_AXI_LAST	CM3_PMU_EV_IOCU_AXI(7)
+#define CM3_PMU_EV_PDTRACE_DROPPED	0x11	/* CM PDTrace dropped messages */
+#define CM3_PMU_EV_PDTRACE_OVF_CYCLES	0x12	/* CM PDTrace overflow cycles */
+#define CM3_PMU_EV_HW_MASK		GENMASK(7, 0)
+/* Pseudo event selecting the cycle counter */
+#define CM3_PMU_EV_CYCLES		0x100
+#define CM3_PMU_EV_MASK			GENMASK(8, 0)
+
+/*
+ * Qualifier of the Coherence Manager Requests event. Ports are numbered
+ * with the cores first, followed by the IOCUs; port bit 8 is the prefetcher
+ * and bit 9 interventions.
+ */
+#define CM3_PMU_Q_CMREQ_PORT		GENMASK_ULL(9, 0)
+#define CM3_PMU_Q_CMREQ_THREAD		GENMASK_ULL(13, 10)
+#define CM3_PMU_Q_CMREQ_TARGET		GENMASK_ULL(15, 14)
+#define  CM3_PMU_Q_CMREQ_TARGET_MEM	BIT_ULL(14)
+#define  CM3_PMU_Q_CMREQ_TARGET_REGBUS	BIT_ULL(15)
+#define CM3_PMU_Q_CMREQ_CMD		GENMASK_ULL(39, 16)
+#define  CM3_PMU_Q_CMREQ_CMD_LEGACY_RD	BIT_ULL(16)
+#define  CM3_PMU_Q_CMREQ_CMD_LEGACY_WR	BIT_ULL(17)
+#define  CM3_PMU_Q_CMREQ_CMD_COH_WB	BIT_ULL(24)
+#define CM3_PMU_Q_CMREQ_CCA		GENMASK_ULL(42, 40) /* UC, UCA, other */
+#define CM3_PMU_Q_CMREQ_SIZE		GENMASK_ULL(44, 43) /* sub-line, line */
+#define CM3_PMU_Q_CMREQ_L1_STATE	GENMASK_ULL(47, 45) /* I, S, E/M */
+#define CM3_PMU_Q_CMREQ_L2_STATE	GENMASK_ULL(51, 48) /* I, S, E, M */
+#define  CM3_PMU_Q_CMREQ_L2_INVALID	BIT_ULL(48)
+#define CM3_PMU_Q_CMREQ_L2_LOCKED	GENMASK_ULL(53, 52)
+#define CM3_PMU_Q_CMREQ_EVICTION	GENMASK_ULL(56, 54)
+#define CM3_PMU_Q_CMREQ_BANK		GENMASK_ULL(58, 57)
+#define CM3_PMU_Q_CMREQ_SCHEDULER	GENMASK_ULL(60, 59)
+#define CM3_PMU_Q_CMREQ_RESERVED	GENMASK_ULL(63, 61) /* must be set */
+#define CM3_PMU_Q_CMREQ_DEFAULT		GENMASK_ULL(63, 0)
+
+/*
+ * Qualifier of the I/O Traffic Requests event. The IOCU number is an
+ * encoded value, not a mask. Direction bits: UC read, coherent read with
+ * allocate, coherent read without allocate, UC write, coherent write.
+ */
+#define CM3_PMU_Q_IO_IOCU		GENMASK_ULL(2, 0)
+#define CM3_PMU_Q_IO_DIR		GENMASK_ULL(7, 3)
+#define CM3_PMU_Q_IO_SIZE		GENMASK_ULL(15, 8)  /* 1 .. 128 bytes */
+#define CM3_PMU_Q_IO_LENGTH		GENMASK_ULL(24, 16) /* 1 .. 129-256 beats */
+#define CM3_PMU_Q_IO_ALIGNED		GENMASK_ULL(26, 25)
+#define CM3_PMU_Q_IO_PREFETCH		GENMASK_ULL(28, 27)
+#define CM3_PMU_Q_IO_ALL_DEVID		BIT_ULL(29)
+#define CM3_PMU_Q_IO_DEVID		GENMASK_ULL(35, 30)
+#define CM3_PMU_Q_IO_ALL_TXNID		BIT_ULL(36)
+#define CM3_PMU_Q_IO_TXNID		GENMASK_ULL(41, 37)
+#define CM3_PMU_Q_IO_DEFAULT		GENMASK_ULL(41, 3)  /* IOCU 0 */
+
+/* Qualifier of the Memory Interface Requests event */
+#define CM3_PMU_Q_MEM_DIR		GENMASK_ULL(1, 0)
+#define  CM3_PMU_Q_MEM_DIR_READ		BIT_ULL(0)
+#define  CM3_PMU_Q_MEM_DIR_WRITE	BIT_ULL(1)
+#define CM3_PMU_Q_MEM_SIZE		GENMASK_ULL(9, 2)   /* 1 .. 128 bytes */
+#define CM3_PMU_Q_MEM_LENGTH		GENMASK_ULL(17, 10) /* 0 .. 7 */
+#define CM3_PMU_Q_MEM_CCA		GENMASK_ULL(21, 18) /* UC, UCA, read discard, cacheable */
+#define CM3_PMU_Q_MEM_SOURCE		GENMASK_ULL(29, 22) /* input port */
+#define CM3_PMU_Q_MEM_THREAD		GENMASK_ULL(35, 32)
+#define CM3_PMU_Q_MEM_CODE_DATA		GENMASK_ULL(37, 36)
+#define CM3_PMU_Q_MEM_PREFETCH		GENMASK_ULL(39, 38)
+#define CM3_PMU_Q_MEM_ALL_GUESTID	BIT_ULL(40)
+#define CM3_PMU_Q_MEM_GUESTID		GENMASK_ULL(45, 41)
+#define CM3_PMU_Q_MEM_DEFAULT		GENMASK_ULL(45, 0)
+
+/* Qualifier of the AXI Bus Utilization events, counts cycles */
+#define CM3_PMU_Q_AXI_NOT_VALID		BIT_ULL(0)
+#define CM3_PMU_Q_AXI_VALID		BIT_ULL(1)
+#define CM3_PMU_Q_AXI_NOT_READY		BIT_ULL(2)
+#define CM3_PMU_Q_AXI_READY		BIT_ULL(3)
+#define CM3_PMU_Q_AXI_CHANNEL		GENMASK_ULL(6, 4)
+#define  CM3_PMU_Q_AXI_CHANNEL_AR	0
+#define  CM3_PMU_Q_AXI_CHANNEL_AW	1
+#define  CM3_PMU_Q_AXI_CHANNEL_W	2
+#define  CM3_PMU_Q_AXI_CHANNEL_R	3
+#define  CM3_PMU_Q_AXI_CHANNEL_B	4
+/* Completed transfers (VALID && READY) on a channel */
+#define CM3_PMU_Q_AXI_XFER(ch)		(FIELD_PREP_CONST(CM3_PMU_Q_AXI_CHANNEL, ch) | \
+					 CM3_PMU_Q_AXI_VALID | CM3_PMU_Q_AXI_READY)
+#define CM3_PMU_Q_AXI_DEFAULT		CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_AR)
+
+/* Qualifier of the CM PDTrace Dropped Messages event */
+#define CM3_PMU_Q_PDTRACE_PORT		GENMASK_ULL(5, 0)
+#define CM3_PMU_Q_PDTRACE_TYPE		GENMASK_ULL(7, 6)   /* request, response */
+#define CM3_PMU_Q_PDTRACE_DEFAULT	GENMASK_ULL(7, 0)
+
+/* Restrict a single qualifier group of a default qualifier to @bits */
+#define CM3_PMU_QUAL(def, group, bits)	(((def) & ~(group)) | (bits))
+
+struct cm3_pmu {
+	struct pmu		pmu;
+	struct hlist_node	node;
+	struct hrtimer		hrtimer;
+	struct perf_event	*events[CM3_PMU_NUM_COUNTERS];
+	struct mutex		init_lock;	/* serializes cm3_pmu_hw_init() */
+	bool			initialized;
+	unsigned int		cluster;
+	unsigned int		num_pcs;
+	unsigned int		num_active;
+	int			on_cpu;
+	int			irq;
+};
+
+struct cm3_pmu_dev {
+	unsigned int		num_pmus;
+	struct cm3_pmu		pmus[] __counted_by(num_pmus);
+};
+
+#define to_cm3_pmu(p)	container_of(p, struct cm3_pmu, pmu)
+
+static int cm3_pmu_cpuhp_state;
+
+/* sysfs attributes */
+
+static ssize_t cm3_pmu_format_show(struct device *dev,
+				   struct device_attribute *attr, char *buf)
+{
+	struct dev_ext_attribute *eattr;
+
+	eattr = container_of(attr, struct dev_ext_attribute, attr);
+	return sysfs_emit(buf, "%s\n", (char *)eattr->var);
+}
+
+#define CM3_PMU_FORMAT_ATTR(_name, _config)				\
+	(&((struct dev_ext_attribute[]) {				\
+		{ .attr = __ATTR(_name, 0444, cm3_pmu_format_show, NULL), \
+		  .var = (void *)_config, }				\
+	})[0].attr.attr)
+
+static struct attribute *cm3_pmu_format_attrs[] = {
+	CM3_PMU_FORMAT_ATTR(event, "config:0-8"),
+	CM3_PMU_FORMAT_ATTR(qual, "config1:0-63"),
+	NULL
+};
+
+static const struct attribute_group cm3_pmu_format_attr_group = {
+	.name = "format",
+	.attrs = cm3_pmu_format_attrs,
+};
+
+struct cm3_pmu_event_attr {
+	struct device_attribute	attr;
+	u16			event;
+	u64			qual;
+};
+
+static ssize_t cm3_pmu_event_show(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	struct cm3_pmu_event_attr *eattr;
+
+	eattr = container_of(attr, struct cm3_pmu_event_attr, attr);
+	if (!eattr->qual)
+		return sysfs_emit(buf, "event=0x%x\n", eattr->event);
+
+	return sysfs_emit(buf, "event=0x%x,qual=0x%llx\n",
+			  eattr->event, eattr->qual);
+}
+
+#define CM3_PMU_EVENT_ATTR(_name, _event, _qual)			\
+	(&((struct cm3_pmu_event_attr[]) {				\
+		{ .attr = __ATTR(_name, 0444, cm3_pmu_event_show, NULL), \
+		  .event = _event, .qual = _qual, }			\
+	})[0].attr.attr)
+
+static struct attribute *cm3_pmu_event_attrs[] = {
+	CM3_PMU_EVENT_ATTR(cycles, CM3_PMU_EV_CYCLES, 0),
+
+	/* Requests serialized by the coherence manager */
+	CM3_PMU_EVENT_ATTR(cm_requests, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_Q_CMREQ_DEFAULT),
+	CM3_PMU_EVENT_ATTR(cm_requests_mem, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+					CM3_PMU_Q_CMREQ_TARGET,
+					CM3_PMU_Q_CMREQ_TARGET_MEM)),
+	CM3_PMU_EVENT_ATTR(cm_requests_regbus, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+					CM3_PMU_Q_CMREQ_TARGET,
+					CM3_PMU_Q_CMREQ_TARGET_REGBUS)),
+	CM3_PMU_EVENT_ATTR(cm_requests_legacy_read, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+					CM3_PMU_Q_CMREQ_CMD,
+					CM3_PMU_Q_CMREQ_CMD_LEGACY_RD)),
+	CM3_PMU_EVENT_ATTR(cm_requests_legacy_write, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+					CM3_PMU_Q_CMREQ_CMD,
+					CM3_PMU_Q_CMREQ_CMD_LEGACY_WR)),
+	CM3_PMU_EVENT_ATTR(cm_requests_coh_writeback, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+					CM3_PMU_Q_CMREQ_CMD,
+					CM3_PMU_Q_CMREQ_CMD_COH_WB)),
+	/* Memory requests whose line is (not) present in the L2 */
+	CM3_PMU_EVENT_ATTR(cm_requests_l2_hit, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+						     CM3_PMU_Q_CMREQ_TARGET,
+						     CM3_PMU_Q_CMREQ_TARGET_MEM),
+					CM3_PMU_Q_CMREQ_L2_STATE,
+					CM3_PMU_Q_CMREQ_L2_STATE &
+					~CM3_PMU_Q_CMREQ_L2_INVALID)),
+	CM3_PMU_EVENT_ATTR(cm_requests_l2_miss, CM3_PMU_EV_CM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_QUAL(CM3_PMU_Q_CMREQ_DEFAULT,
+						     CM3_PMU_Q_CMREQ_TARGET,
+						     CM3_PMU_Q_CMREQ_TARGET_MEM),
+					CM3_PMU_Q_CMREQ_L2_STATE,
+					CM3_PMU_Q_CMREQ_L2_INVALID)),
+
+	/* Requests received by IOCU 0 */
+	CM3_PMU_EVENT_ATTR(iocu0_requests, CM3_PMU_EV_IO_REQ,
+			   CM3_PMU_Q_IO_DEFAULT),
+
+	/* Requests issued on the memory interface */
+	CM3_PMU_EVENT_ATTR(mem_requests, CM3_PMU_EV_MEM_REQ,
+			   CM3_PMU_Q_MEM_DEFAULT),
+	CM3_PMU_EVENT_ATTR(mem_reads, CM3_PMU_EV_MEM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_MEM_DEFAULT,
+					CM3_PMU_Q_MEM_DIR,
+					CM3_PMU_Q_MEM_DIR_READ)),
+	CM3_PMU_EVENT_ATTR(mem_writes, CM3_PMU_EV_MEM_REQ,
+			   CM3_PMU_QUAL(CM3_PMU_Q_MEM_DEFAULT,
+					CM3_PMU_Q_MEM_DIR,
+					CM3_PMU_Q_MEM_DIR_WRITE)),
+
+	/* Transfers on each channel of the memory AXI bus */
+	CM3_PMU_EVENT_ATTR(mem_axi_ar, CM3_PMU_EV_MEM_AXI,
+			   CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_AR)),
+	CM3_PMU_EVENT_ATTR(mem_axi_aw, CM3_PMU_EV_MEM_AXI,
+			   CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_AW)),
+	CM3_PMU_EVENT_ATTR(mem_axi_w, CM3_PMU_EV_MEM_AXI,
+			   CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_W)),
+	CM3_PMU_EVENT_ATTR(mem_axi_r, CM3_PMU_EV_MEM_AXI,
+			   CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_R)),
+	CM3_PMU_EVENT_ATTR(mem_axi_b, CM3_PMU_EV_MEM_AXI,
+			   CM3_PMU_Q_AXI_XFER(CM3_PMU_Q_AXI_CHANNEL_B)),
+
+	CM3_PMU_EVENT_ATTR(pdtrace_dropped, CM3_PMU_EV_PDTRACE_DROPPED,
+			   CM3_PMU_Q_PDTRACE_DEFAULT),
+	CM3_PMU_EVENT_ATTR(pdtrace_overflow_cycles,
+			   CM3_PMU_EV_PDTRACE_OVF_CYCLES, 0),
+	NULL
+};
+
+static const struct attribute_group cm3_pmu_events_attr_group = {
+	.name = "events",
+	.attrs = cm3_pmu_event_attrs,
+};
+
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
+			    char *buf)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(dev_get_drvdata(dev));
+
+	return sysfs_emit(buf, "%*pbl\n",
+			  cpumask_pr_args(cpumask_of(cm3_pmu->on_cpu)));
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *cm3_pmu_cpumask_attrs[] = {
+	&dev_attr_cpumask.attr,
+	NULL
+};
+
+static const struct attribute_group cm3_pmu_cpumask_attr_group = {
+	.attrs = cm3_pmu_cpumask_attrs,
+};
+
+static const struct attribute_group *cm3_pmu_attr_groups[] = {
+	&cm3_pmu_format_attr_group,
+	&cm3_pmu_events_attr_group,
+	&cm3_pmu_cpumask_attr_group,
+	NULL
+};
+
+/*
+ * Hardware access
+ *
+ * The debug block of the local cluster is accessed directly. That of another
+ * cluster is accessed through the redirect block, pointed at the debug block
+ * of the target cluster, from whichever CPU this runs on. Both paths are
+ * valid in any context; the redirect path takes the per-CPU redirect lock
+ * with interrupts disabled.
+ */
+
+static bool cm3_pmu_lock(struct cm3_pmu *cm3_pmu)
+{
+	preempt_disable();
+	if (cm3_pmu->cluster == cpu_cluster(&current_cpu_data))
+		return false;
+
+	mips_cm_lock_other(cm3_pmu->cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_DEBUG);
+	return true;
+}
+
+static void cm3_pmu_unlock(bool other)
+{
+	if (other)
+		mips_cm_unlock_other();
+	preempt_enable();
+}
+
+#define CM3_PMU_READ(cm3_pmu, reg)					\
+({									\
+	bool __other = cm3_pmu_lock(cm3_pmu);				\
+	typeof(read_gcr_db_##reg()) __val;				\
+									\
+	__val = __other ? read_gcr_redir_db_##reg() : read_gcr_db_##reg(); \
+	cm3_pmu_unlock(__other);					\
+	__val;								\
+})
+
+#define CM3_PMU_WRITE(cm3_pmu, reg, val)				\
+do {									\
+	typeof(read_gcr_db_##reg()) __val = (val);			\
+	bool __other = cm3_pmu_lock(cm3_pmu);				\
+									\
+	if (__other)							\
+		write_gcr_redir_db_##reg(__val);			\
+	else								\
+		write_gcr_db_##reg(__val);				\
+	cm3_pmu_unlock(__other);					\
+} while (0)
+
+/*
+ * Whether the CM of the cluster is powered. The CPC is always powered and
+ * reports the CM power sequencer state in the STAT_CONF of its "CM core".
+ * Accessing the GCRs of a powered down cluster would fault.
+ */
+static bool cm3_pmu_cluster_powered(struct cm3_pmu *cm3_pmu)
+{
+	u32 stat;
+
+	/* No other cluster before CM 3.5, and ours is obviously powered */
+	if (cm3_pmu->cluster == cpu_cluster(&raw_current_cpu_data))
+		return true;
+
+	if (!mips_cpc_present())
+		return false;
+
+	mips_cm_lock_other(cm3_pmu->cluster, CM_GCR_Cx_OTHER_CORE_CM, 0,
+			   CM_GCR_Cx_OTHER_BLOCK_LOCAL);
+	stat = read_cpc_co_stat_conf();
+	mips_cm_unlock_other();
+
+	return FIELD_GET(CPC_Cx_STAT_CONF_SEQSTATE, stat) ==
+	       CPC_Cx_STAT_CONF_SEQSTATE_U5;
+}
+
+static u32 cm3_pmu_counton_bit(int idx)
+{
+	switch (idx) {
+	case 0:
+		return CM_GCR_DB_PC_CTL_P0_COUNTON;
+	case 1:
+		return CM_GCR_DB_PC_CTL_P1_COUNTON;
+	default:
+		return CM_GCR_DB_PC_CTL_CYCL_CNT_COUNTON;
+	}
+}
+
+static u32 cm3_pmu_reset_bit(int idx)
+{
+	switch (idx) {
+	case 0:
+		return CM_GCR_DB_PC_CTL_P0_RESET;
+	case 1:
+		return CM_GCR_DB_PC_CTL_P1_RESET;
+	default:
+		return CM_GCR_DB_PC_CTL_CYCL_CNT_RESET;
+	}
+}
+
+static u32 cm3_pmu_overflow_bit(int idx)
+{
+	switch (idx) {
+	case 0:
+		return CM_GCR_DB_PC_OV_P0_OF;
+	case 1:
+		return CM_GCR_DB_PC_OV_P1_OF;
+	default:
+		return CM_GCR_DB_PC_OV_CYCL_CNT_OF;
+	}
+}
+
+static u32 cm3_pmu_read_counter(struct cm3_pmu *cm3_pmu, int idx)
+{
+	switch (idx) {
+	case 0:
+		return CM3_PMU_READ(cm3_pmu, pc_cnt0);
+	case 1:
+		return CM3_PMU_READ(cm3_pmu, pc_cnt1);
+	default:
+		return CM3_PMU_READ(cm3_pmu, pc_cycl);
+	}
+}
+
+static void cm3_pmu_program_counter(struct cm3_pmu *cm3_pmu, int idx,
+				    u32 event, u64 qual)
+{
+	u32 val = CM3_PMU_READ(cm3_pmu, pc_event);
+
+	if (idx == 0) {
+		val &= ~CM_GCR_DB_PC_EVENT_P0_EVENT;
+		val |= FIELD_PREP(CM_GCR_DB_PC_EVENT_P0_EVENT, event);
+		CM3_PMU_WRITE(cm3_pmu, pc_event, val);
+		CM3_PMU_WRITE(cm3_pmu, pc_qual0, qual);
+	} else {
+		val &= ~CM_GCR_DB_PC_EVENT_P1_EVENT;
+		val |= FIELD_PREP(CM_GCR_DB_PC_EVENT_P1_EVENT, event);
+		CM3_PMU_WRITE(cm3_pmu, pc_event, val);
+		CM3_PMU_WRITE(cm3_pmu, pc_qual1, qual);
+	}
+}
+
+static void cm3_pmu_set_ctl(struct cm3_pmu *cm3_pmu, u32 clear, u32 set)
+{
+	u32 val = CM3_PMU_READ(cm3_pmu, pc_ctl);
+
+	CM3_PMU_WRITE(cm3_pmu, pc_ctl, (val & ~clear) | set);
+}
+
+/* Stop everything, clear pending overflows, enable the interrupt if used */
+static void cm3_pmu_hw_reset(struct cm3_pmu *cm3_pmu)
+{
+	CM3_PMU_WRITE(cm3_pmu, pc_ctl,
+		      cm3_pmu->irq > 0 ? CM_GCR_DB_PC_CTL_PERF_INT_EN : 0);
+	CM3_PMU_WRITE(cm3_pmu, pc_event, 0);
+	CM3_PMU_WRITE(cm3_pmu, pc_ov, CM_GCR_DB_PC_OV_P1_OF |
+				      CM_GCR_DB_PC_OV_P0_OF |
+				      CM_GCR_DB_PC_OV_CYCL_CNT_OF);
+}
+
+/*
+ * Bring the counters of the cluster to a known state the first time they
+ * are needed. The CM may not have been powered at probe time.
+ */
+static int cm3_pmu_hw_init(struct cm3_pmu *cm3_pmu)
+{
+	guard(mutex)(&cm3_pmu->init_lock);
+
+	if (cm3_pmu->initialized)
+		return 0;
+
+	if (!cm3_pmu_cluster_powered(cm3_pmu))
+		return -ENODEV;
+
+	cm3_pmu->num_pcs = FIELD_GET(CM_GCR_DB_PC_CTL_PERF_NUM_CNT,
+				     CM3_PMU_READ(cm3_pmu, pc_ctl));
+	cm3_pmu->num_pcs = min_t(unsigned int, cm3_pmu->num_pcs,
+				 CM3_PMU_MAX_PCS);
+	cm3_pmu_hw_reset(cm3_pmu);
+	cm3_pmu->initialized = true;
+
+	return 0;
+}
+
+static u64 cm3_pmu_default_qual(u32 event)
+{
+	switch (event) {
+	case CM3_PMU_EV_CM_REQ:
+		return CM3_PMU_Q_CMREQ_DEFAULT;
+	case CM3_PMU_EV_IO_REQ:
+		return CM3_PMU_Q_IO_DEFAULT;
+	case CM3_PMU_EV_MEM_REQ:
+		return CM3_PMU_Q_MEM_DEFAULT;
+	case CM3_PMU_EV_MEM_AXI:
+	case CM3_PMU_EV_IOCU_AXI(0) ... CM3_PMU_EV_IOCU_AXI_LAST:
+		return CM3_PMU_Q_AXI_DEFAULT;
+	case CM3_PMU_EV_PDTRACE_DROPPED:
+		return CM3_PMU_Q_PDTRACE_DEFAULT;
+	default:
+		/* All qualifier groups set to "don't care" */
+		return GENMASK_ULL(63, 0);
+	}
+}
+
+/* perf callbacks */
+
+static void cm3_pmu_update(struct perf_event *event)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct hw_perf_event *hwc = &event->hw;
+	u64 prev, new;
+
+	do {
+		prev = local64_read(&hwc->prev_count);
+		new = cm3_pmu_read_counter(cm3_pmu, hwc->idx);
+	} while (local64_cmpxchg(&hwc->prev_count, prev, new) != prev);
+
+	local64_add((new - prev) & CM3_PMU_COUNTER_MASK, &event->count);
+}
+
+static void cm3_pmu_start(struct perf_event *event, int flags)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct hw_perf_event *hwc = &event->hw;
+	int idx = hwc->idx;
+
+	if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED)))
+		return;
+
+	hwc->state = 0;
+
+	if (idx != CM3_PMU_IDX_CYCLES)
+		cm3_pmu_program_counter(cm3_pmu, idx, hwc->config,
+					hwc->config_base);
+
+	/*
+	 * Setting the reset bit together with the count-on bit resets the
+	 * counter and its overflow status before counting is started.
+	 */
+	local64_set(&hwc->prev_count, 0);
+	cm3_pmu_set_ctl(cm3_pmu, 0,
+			cm3_pmu_reset_bit(idx) | cm3_pmu_counton_bit(idx));
+
+	perf_event_update_userpage(event);
+}
+
+static void cm3_pmu_stop(struct perf_event *event, int flags)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct hw_perf_event *hwc = &event->hw;
+
+	if (hwc->state & PERF_HES_STOPPED)
+		return;
+
+	cm3_pmu_set_ctl(cm3_pmu, cm3_pmu_counton_bit(hwc->idx), 0);
+
+	cm3_pmu_update(event);
+	hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
+}
+
+static int cm3_pmu_add(struct perf_event *event, int flags)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct hw_perf_event *hwc = &event->hw;
+	int idx;
+
+	if (hwc->config == CM3_PMU_EV_CYCLES) {
+		idx = CM3_PMU_IDX_CYCLES;
+		if (cm3_pmu->events[idx])
+			return -EAGAIN;
+	} else {
+		for (idx = 0; idx < cm3_pmu->num_pcs; idx++)
+			if (!cm3_pmu->events[idx])
+				break;
+		if (idx == cm3_pmu->num_pcs)
+			return -EAGAIN;
+	}
+
+	hwc->idx = idx;
+	hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE;
+	cm3_pmu->events[idx] = event;
+
+	if (!cm3_pmu->num_active++ && cm3_pmu->irq <= 0)
+		hrtimer_start(&cm3_pmu->hrtimer,
+			      ns_to_ktime(CM3_PMU_POLL_PERIOD_NS),
+			      HRTIMER_MODE_REL_PINNED);
+
+	if (flags & PERF_EF_START)
+		cm3_pmu_start(event, PERF_EF_RELOAD);
+
+	perf_event_update_userpage(event);
+
+	return 0;
+}
+
+static void cm3_pmu_del(struct perf_event *event, int flags)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct hw_perf_event *hwc = &event->hw;
+
+	cm3_pmu_stop(event, PERF_EF_UPDATE);
+
+	cm3_pmu->events[hwc->idx] = NULL;
+	if (!--cm3_pmu->num_active && cm3_pmu->irq <= 0)
+		hrtimer_cancel(&cm3_pmu->hrtimer);
+
+	perf_event_update_userpage(event);
+}
+
+static void cm3_pmu_count_group_event(struct perf_event *event,
+				      struct perf_event *member,
+				      unsigned int *cycles, unsigned int *pcs)
+{
+	if (member->pmu != event->pmu)
+		return;
+
+	if (member->hw.config == CM3_PMU_EV_CYCLES)
+		(*cycles)++;
+	else
+		(*pcs)++;
+}
+
+static bool cm3_pmu_validate_group(struct perf_event *event)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct perf_event *leader = event->group_leader;
+	struct perf_event *sibling;
+	unsigned int cycles = 0, pcs = 0;
+
+	/* Hardware events of the group must all be ours */
+	if (leader->pmu != event->pmu && !is_software_event(leader))
+		return false;
+
+	cm3_pmu_count_group_event(event, event, &cycles, &pcs);
+	if (leader != event)
+		cm3_pmu_count_group_event(event, leader, &cycles, &pcs);
+
+	for_each_sibling_event(sibling, leader) {
+		if (sibling->pmu != event->pmu && !is_software_event(sibling))
+			return false;
+
+		cm3_pmu_count_group_event(event, sibling, &cycles, &pcs);
+	}
+
+	return cycles <= 1 && pcs <= cm3_pmu->num_pcs;
+}
+
+static int cm3_pmu_event_init(struct perf_event *event)
+{
+	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
+	struct hw_perf_event *hwc = &event->hw;
+	u64 config = event->attr.config;
+	int ret;
+
+	if (event->attr.type != event->pmu->type)
+		return -ENOENT;
+
+	/* Counters are shared by all CPUs: no sampling, no per-task events */
+	if (is_sampling_event(event) ||
+	    event->attach_state & PERF_ATTACH_TASK)
+		return -EOPNOTSUPP;
+
+	if (event->cpu < 0)
+		return -EINVAL;
+
+	if (config & ~CM3_PMU_EV_MASK)
+		return -EINVAL;
+
+	if (config != CM3_PMU_EV_CYCLES &&
+	    (config & ~CM3_PMU_EV_HW_MASK || config == CM3_PMU_EV_NONE))
+		return -EINVAL;
+
+	/* The cluster's CM must be powered for its counters to be usable */
+	ret = cm3_pmu_hw_init(cm3_pmu);
+	if (ret)
+		return ret;
+
+	if (!cm3_pmu_validate_group(event))
+		return -EINVAL;
+
+	hwc->idx = -1;
+	hwc->config = config;
+	hwc->config_base = event->attr.config1 ?: cm3_pmu_default_qual(config);
+	event->cpu = cm3_pmu->on_cpu;
+
+	return 0;
+}
+
+static enum hrtimer_restart cm3_pmu_poll(struct hrtimer *hrtimer)
+{
+	struct cm3_pmu *cm3_pmu = container_of(hrtimer, struct cm3_pmu,
+					       hrtimer);
+	int idx;
+
+	for (idx = 0; idx < CM3_PMU_NUM_COUNTERS; idx++) {
+		struct perf_event *event = cm3_pmu->events[idx];
+
+		if (event && !(event->hw.state & PERF_HES_STOPPED))
+			cm3_pmu_update(event);
+	}
+
+	hrtimer_forward_now(hrtimer, ns_to_ktime(CM3_PMU_POLL_PERIOD_NS));
+
+	return HRTIMER_RESTART;
+}
+
+static irqreturn_t cm3_pmu_handle_irq(int irq, void *data)
+{
+	struct cm3_pmu *cm3_pmu = data;
+	u32 ov = CM3_PMU_READ(cm3_pmu, pc_ov);
+	int idx;
+
+	if (!ov)
+		return IRQ_NONE;
+
+	/* Status bits are write-1-to-clear, this deasserts the interrupt */
+	CM3_PMU_WRITE(cm3_pmu, pc_ov, ov);
+
+	for (idx = 0; idx < CM3_PMU_NUM_COUNTERS; idx++) {
+		struct perf_event *event = cm3_pmu->events[idx];
+
+		if (event && ov & cm3_pmu_overflow_bit(idx))
+			cm3_pmu_update(event);
+	}
+
+	return IRQ_HANDLED;
+}
+
+/* CPU hotplug */
+
+static int cm3_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct cm3_pmu *cm3_pmu = hlist_entry_safe(node, struct cm3_pmu, node);
+
+	if (cm3_pmu->on_cpu >= 0)
+		return 0;
+
+	cm3_pmu->on_cpu = cpu;
+	if (cm3_pmu->irq > 0)
+		WARN_ON(irq_set_affinity(cm3_pmu->irq, cpumask_of(cpu)));
+
+	return 0;
+}
+
+static int cm3_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct cm3_pmu *cm3_pmu = hlist_entry_safe(node, struct cm3_pmu, node);
+	unsigned int target;
+
+	if (cm3_pmu->on_cpu != cpu)
+		return 0;
+
+	target = cpumask_any_but(cpu_online_mask, cpu);
+	if (target >= nr_cpu_ids) {
+		cm3_pmu->on_cpu = -1;
+		return 0;
+	}
+
+	perf_pmu_migrate_context(&cm3_pmu->pmu, cpu, target);
+	cm3_pmu->on_cpu = target;
+	if (cm3_pmu->irq > 0)
+		WARN_ON(irq_set_affinity(cm3_pmu->irq, cpumask_of(target)));
+
+	return 0;
+}
+
+/* Platform driver */
+
+static void cm3_pmu_unregister(struct cm3_pmu *cm3_pmu)
+{
+	perf_pmu_unregister(&cm3_pmu->pmu);
+	cpuhp_state_remove_instance(cm3_pmu_cpuhp_state, &cm3_pmu->node);
+	hrtimer_cancel(&cm3_pmu->hrtimer);
+	if (cm3_pmu->initialized && cm3_pmu_cluster_powered(cm3_pmu)) {
+		cm3_pmu->irq = 0;
+		cm3_pmu_hw_reset(cm3_pmu);
+	}
+}
+
+static int cm3_pmu_register(struct device *dev, struct cm3_pmu *cm3_pmu)
+{
+	const char *name;
+	int ret;
+
+	cm3_pmu->on_cpu = -1;
+	cm3_pmu->num_pcs = CM3_PMU_MAX_PCS;
+	mutex_init(&cm3_pmu->init_lock);
+	hrtimer_setup(&cm3_pmu->hrtimer, cm3_pmu_poll, CLOCK_MONOTONIC,
+		      HRTIMER_MODE_REL);
+
+	cm3_pmu->pmu = (struct pmu) {
+		.parent		= dev,
+		.module		= THIS_MODULE,
+		.task_ctx_nr	= perf_invalid_context,
+		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE |
+				  PERF_PMU_CAP_NO_INTERRUPT,
+		.event_init	= cm3_pmu_event_init,
+		.add		= cm3_pmu_add,
+		.del		= cm3_pmu_del,
+		.start		= cm3_pmu_start,
+		.stop		= cm3_pmu_stop,
+		.read		= cm3_pmu_update,
+		.attr_groups	= cm3_pmu_attr_groups,
+	};
+
+	name = devm_kasprintf(dev, GFP_KERNEL, CM3_PMU_NAME "_%u",
+			      cm3_pmu->cluster);
+	if (!name)
+		return -ENOMEM;
+
+	/* Initialize now if the cluster is powered, else on first use */
+	ret = cm3_pmu_hw_init(cm3_pmu);
+	if (ret && ret != -ENODEV)
+		return ret;
+
+	ret = cpuhp_state_add_instance(cm3_pmu_cpuhp_state, &cm3_pmu->node);
+	if (ret)
+		return dev_err_probe(dev, ret, "failed to register hotplug\n");
+
+	ret = perf_pmu_register(&cm3_pmu->pmu, name, -1);
+	if (ret) {
+		cpuhp_state_remove_instance(cm3_pmu_cpuhp_state, &cm3_pmu->node);
+		return dev_err_probe(dev, ret, "failed to register %s\n", name);
+	}
+
+	if (cm3_pmu->initialized)
+		dev_info(dev, "%s: %u counters + cycles, %s\n", name,
+			 cm3_pmu->num_pcs,
+			 cm3_pmu->irq > 0 ? "overflow interrupt" : "polled");
+	else
+		dev_info(dev, "%s: cluster powered down\n", name);
+
+	return 0;
+}
+
+static int cm3_pmu_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct cm3_pmu_dev *cm3_dev;
+	unsigned int nclusters, i;
+	int irq, ret;
+
+	if (!mips_cm_present() || mips_cm_revision() < CM_REV_CM3)
+		return -ENODEV;
+
+	/* One PMU per cluster, whether or not it holds CPUs */
+	nclusters = mips_cps_numclusters();
+	cm3_dev = devm_kzalloc(dev, struct_size(cm3_dev, pmus, nclusters),
+			       GFP_KERNEL);
+	if (!cm3_dev)
+		return -ENOMEM;
+
+	cm3_dev->num_pmus = nclusters;
+	for (i = 0; i < nclusters; i++)
+		cm3_dev->pmus[i].cluster = i;
+
+	/*
+	 * Each cluster's CM raises its own overflow interrupt into its own
+	 * GIC, and a GIC shared interrupt is routed to a single cluster at a
+	 * time (the one of its affinity CPU). The interrupt can therefore
+	 * only serve the cluster the CPUs live in, and only if they all live
+	 * in that one cluster so that any CPU may handle it. Other clusters
+	 * are polled.
+	 */
+	irq = platform_get_irq_optional(pdev, 0);
+	if (irq > 0 && !mips_cps_multicluster_cpus()) {
+		struct cm3_pmu *cm3_pmu;
+
+		cm3_pmu = &cm3_dev->pmus[cpu_cluster(&boot_cpu_data)];
+		ret = devm_request_irq(dev, irq, cm3_pmu_handle_irq,
+				       IRQF_NOBALANCING | IRQF_NO_THREAD,
+				       CM3_PMU_NAME, cm3_pmu);
+		if (ret)
+			return dev_err_probe(dev, ret, "failed to request IRQ\n");
+
+		cm3_pmu->irq = irq;
+	} else if (irq > 0) {
+		dev_info(dev, "CPUs in several clusters: overflow interrupt not usable, polling\n");
+	} else if (irq != -ENXIO) {
+		return dev_err_probe(dev, irq, "failed to get IRQ\n");
+	}
+
+	for (i = 0; i < cm3_dev->num_pmus; i++) {
+		ret = cm3_pmu_register(dev, &cm3_dev->pmus[i]);
+		if (ret)
+			goto err_unregister;
+	}
+
+	platform_set_drvdata(pdev, cm3_dev);
+
+	return 0;
+
+err_unregister:
+	while (i--)
+		cm3_pmu_unregister(&cm3_dev->pmus[i]);
+
+	return ret;
+}
+
+static void cm3_pmu_remove(struct platform_device *pdev)
+{
+	struct cm3_pmu_dev *cm3_dev = platform_get_drvdata(pdev);
+	unsigned int i;
+
+	for (i = 0; i < cm3_dev->num_pmus; i++)
+		cm3_pmu_unregister(&cm3_dev->pmus[i]);
+}
+
+static const struct of_device_id cm3_pmu_of_match[] = {
+	{ .compatible = "mti,mips-cm" },
+	{ .compatible = "mobileye,eyeq6-cm" },
+	{}
+};
+MODULE_DEVICE_TABLE(of, cm3_pmu_of_match);
+
+static struct platform_driver cm3_pmu_driver = {
+	.driver = {
+		.name = CM3_PMU_NAME,
+		.of_match_table = cm3_pmu_of_match,
+		.suppress_bind_attrs = true,
+	},
+	.probe = cm3_pmu_probe,
+	.remove = cm3_pmu_remove,
+};
+
+static int __init cm3_pmu_init(void)
+{
+	int ret;
+
+	ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+				      "perf/mips/cm3_pmu:online",
+				      cm3_pmu_online_cpu, cm3_pmu_offline_cpu);
+	if (ret < 0)
+		return ret;
+
+	cm3_pmu_cpuhp_state = ret;
+
+	ret = platform_driver_register(&cm3_pmu_driver);
+	if (ret)
+		cpuhp_remove_multi_state(cm3_pmu_cpuhp_state);
+
+	return ret;
+}
+module_init(cm3_pmu_init);
+
+static void __exit cm3_pmu_exit(void)
+{
+	platform_driver_unregister(&cm3_pmu_driver);
+	cpuhp_remove_multi_state(cm3_pmu_cpuhp_state);
+}
+module_exit(cm3_pmu_exit);
+
+MODULE_DESCRIPTION("MIPS Coherence Manager 3 (CM3) PMU driver");
+MODULE_LICENSE("GPL");

-- 
2.43.0

[PATCH 5/5] MIPS: eyeq6_defconfig: enable the CM3 PMU

[Vladimir Kondratiev]

Enable perf events and build the Coherence Manager 3 PMU driver as a
module.

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
---
 arch/mips/configs/eyeq6_defconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/mips/configs/eyeq6_defconfig b/arch/mips/configs/eyeq6_defconfig
index 0a00a201937b..2d06d9ed6526 100644
--- a/arch/mips/configs/eyeq6_defconfig
+++ b/arch/mips/configs/eyeq6_defconfig
@@ -19,6 +19,7 @@ CONFIG_USER_NS=y
 CONFIG_SCHED_AUTOGROUP=y
 CONFIG_BLK_DEV_INITRD=y
 CONFIG_EXPERT=y
+CONFIG_PERF_EVENTS=y
 CONFIG_EYEQ=y
 CONFIG_MACH_EYEQ6H=y
 CONFIG_MIPS_CPS=y
@@ -86,6 +87,7 @@ CONFIG_MMC_SDHCI_PLTFM=y
 CONFIG_MMC_SDHCI_CADENCE=y
 # CONFIG_IOMMU_SUPPORT is not set
 CONFIG_RESET_CONTROLLER=y
+CONFIG_MIPS_CM3_PMU=m
 # CONFIG_NVMEM is not set
 CONFIG_EXT4_FS=y
 CONFIG_EXT4_FS_POSIX_ACL=y

-- 
2.43.0

Re: [PATCH 4/5] perf: add MIPS Coherence Manager 3 (CM3) PMU driver

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 7 potential
issues to consider:
- [High] perf/mips_cm3: 2^32 counts lost on counter overflow in
  cm3_pmu_update()
- [Medium] perf/mips_cm3: 64-bit qualifier truncated in hwc->config_base
  on 32-bit
- [Medium] perf/mips_cm3: cm3_pmu_event_init() validates group before
  setting config
- [Medium] perf/mips_cm3: NULL ctx deref in cm3_pmu_validate_group()
  with lockdep
- [Medium] perf/mips_cm3: cpuhp_state_remove_instance() migrates
  unregistered PMU
- [Medium] perf/mips_cm3: cm3_pmu_hw_init() skips power check when
  initialized
- [Medium] perf/mips_cm3: invalid wait context acquiring cm_core_lock in
  cm3_pmu_lock()
--

commit 7d10c5a1b5055212bfd02b13d5581ab06e3a9b14
Author: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>

perf: add MIPS Coherence Manager 3 (CM3) PMU driver

This commit adds an uncore PMU driver and documentation for the performance
counters in the MIPS Coherence Manager 3 (CM3).

> diff --git a/drivers/perf/mips_cm3_pmu.c b/drivers/perf/mips_cm3_pmu.c
> new file mode 100644
> index 0000000000000..d4be23e79d6f0
> --- /dev/null
> +++ b/drivers/perf/mips_cm3_pmu.c

[ ... ]

> +static bool cm3_pmu_lock(struct cm3_pmu *cm3_pmu)
> +{
> +	preempt_disable();
> +	if (cm3_pmu->cluster == cpu_cluster(&current_cpu_data))
> +		return false;
> +
> +	mips_cm_lock_other(cm3_pmu->cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_DEBUG);
> +	return true;
> +}

[Severity: Medium]
Can calling mips_cm_lock_other() here trigger a lockdep wait-context
splat under CONFIG_PROVE_RAW_LOCK_NESTING?

mips_cm_lock_other() acquires the per-CPU cm_core_lock, which is a
non-raw spinlock_t:

arch/mips/kernel/mips-cm.c:mips_cm_lock_other() {
	...
	spin_lock_irqsave(this_cpu_ptr(&cm_core_lock),
			  *this_cpu_ptr(&cm_core_lock_flags));
	...
}

CM3_PMU_READ() and CM3_PMU_WRITE() call cm3_pmu_lock() from the PMU
callbacks (add, start, stop, del, read), which run while holding the
raw_spinlock_t perf_event_context::lock, as well as from the hardirq
hrtimer callback cm3_pmu_poll().  Acquiring a spinlock_t inside a
raw_spinlock_t or hardirq context is an invalid wait-context transition
when accessing a remote cluster's PMU.

[ ... ]

> +static int cm3_pmu_hw_init(struct cm3_pmu *cm3_pmu)
> +{
> +	guard(mutex)(&cm3_pmu->init_lock);
> +
> +	if (cm3_pmu->initialized)
> +		return 0;
> +
> +	if (!cm3_pmu_cluster_powered(cm3_pmu))
> +		return -ENODEV;
> +
> +	cm3_pmu->num_pcs = FIELD_GET(CM_GCR_DB_PC_CTL_PERF_NUM_CNT,
> +				     CM3_PMU_READ(cm3_pmu, pc_ctl));
> +	cm3_pmu->num_pcs = min_t(unsigned int, cm3_pmu->num_pcs,
> +				 CM3_PMU_MAX_PCS);
> +	cm3_pmu_hw_reset(cm3_pmu);
> +	cm3_pmu->initialized = true;
> +
> +	return 0;
> +}

[Severity: Medium]
Does returning early when cm3_pmu->initialized is true skip the power
check if a remote cluster is powered down after probe or an earlier
event?  As cm3_pmu_unregister() notes by checking both
cm3_pmu->initialized and cm3_pmu_cluster_powered(), a cluster that was
once initialized can later power down; opening a new event on it will
then pass cm3_pmu_hw_init() and fault when cm3_pmu_start() accesses its
GCRs.

[ ... ]

> +static void cm3_pmu_update(struct perf_event *event)
> +{
> +	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
> +	struct hw_perf_event *hwc = &event->hw;
> +	u64 prev, new;
> +
> +	do {
> +		prev = local64_read(&hwc->prev_count);
> +		new = cm3_pmu_read_counter(cm3_pmu, hwc->idx);
> +	} while (local64_cmpxchg(&hwc->prev_count, prev, new) != prev);
> +
> +	local64_add((new - prev) & CM3_PMU_COUNTER_MASK, &event->count);
> +}
> +
> +static void cm3_pmu_start(struct perf_event *event, int flags)
> +{
> +	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
> +	struct hw_perf_event *hwc = &event->hw;
> +	int idx = hwc->idx;
> +
> +	if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED)))
> +		return;
> +
> +	hwc->state = 0;
> +
> +	if (idx != CM3_PMU_IDX_CYCLES)
> +		cm3_pmu_program_counter(cm3_pmu, idx, hwc->config,
> +					hwc->config_base);
> +
> +	/*
> +	 * Setting the reset bit together with the count-on bit resets the
> +	 * counter and its overflow status before counting is started.
> +	 */
> +	local64_set(&hwc->prev_count, 0);
> +	cm3_pmu_set_ctl(cm3_pmu, 0,
> +			cm3_pmu_reset_bit(idx) | cm3_pmu_counton_bit(idx));
> +
> +	perf_event_update_userpage(event);
> +}

[Severity: High]
When operating in overflow-interrupt mode (cm3_pmu->irq > 0, where the
polling timer is not started), does cm3_pmu_update() lose 2^32 counts on
counter wraparound?

cm3_pmu_start() resets both hwc->prev_count and the 32-bit hardware
counter to 0.  When the counter reaches 0xffffffff, raises the overflow
interrupt, and wraps past 0 to a small value new before
cm3_pmu_handle_irq() calls cm3_pmu_update(), (new - prev) & 0xffffffff
evaluates to just new (or on subsequent overflows, whenever new >= prev),
discarding the full 2^32 wraparound.

[ ... ]

> +static void cm3_pmu_count_group_event(struct perf_event *event,
> +				      struct perf_event *member,
> +				      unsigned int *cycles, unsigned int *pcs)
> +{
> +	if (member->pmu != event->pmu)
> +		return;
> +
> +	if (member->hw.config == CM3_PMU_EV_CYCLES)
> +		(*cycles)++;
> +	else
> +		(*pcs)++;
> +}
> +
> +static bool cm3_pmu_validate_group(struct perf_event *event)
> +{
> +	struct cm3_pmu *cm3_pmu = to_cm3_pmu(event->pmu);
> +	struct perf_event *leader = event->group_leader;
> +	struct perf_event *sibling;
> +	unsigned int cycles = 0, pcs = 0;
> +
> +	/* Hardware events of the group must all be ours */
> +	if (leader->pmu != event->pmu && !is_software_event(leader))
> +		return false;
> +
> +	cm3_pmu_count_group_event(event, event, &cycles, &pcs);
> +	if (leader != event)
> +		cm3_pmu_count_group_event(event, leader, &cycles, &pcs);
> +
> +	for_each_sibling_event(sibling, leader) {
> +		if (sibling->pmu != event->pmu && !is_software_event(sibling))
> +			return false;
> +
> +		cm3_pmu_count_group_event(event, sibling, &cycles, &pcs);
> +	}
> +
> +	return cycles <= 1 && pcs <= cm3_pmu->num_pcs;
> +}

[Severity: Medium]
Can for_each_sibling_event(sibling, leader) cause a NULL pointer
dereference when leader == event and CONFIG_PROVE_LOCKING is enabled?

When initializing a standalone event or group leader in
perf_try_init_event(), leader->ctx is still NULL, and
for_each_sibling_event() expands lockdep_assert_event_ctx(leader), which
dereferences leader->ctx->mutex.  Returning true before the loop when
leader == event avoids this.

> +
> +static int cm3_pmu_event_init(struct perf_event *event)
> +{

[ ... ]

> +	if (!cm3_pmu_validate_group(event))
> +		return -EINVAL;
> +
> +	hwc->idx = -1;
> +	hwc->config = config;

[Severity: Medium]
Is cm3_pmu_validate_group(event) called before hwc->config is set?

Because event->hw.config is still 0 during cm3_pmu_validate_group(),
cm3_pmu_count_group_event(event, event, &cycles, &pcs) always counts the
new event in pcs instead of cycles.  Adding a CM3_PMU_EV_CYCLES event to
a group that already has two programmable events fails with -EINVAL,
while a group with two CM3_PMU_EV_CYCLES events passes validation.

> +	hwc->config_base = event->attr.config1 ?: cm3_pmu_default_qual(config);
> +	event->cpu = cm3_pmu->on_cpu;
> +
> +	return 0;
> +}

[Severity: Medium]
Does storing the 64-bit qualifier in hwc->config_base truncate bits 63:32
on 32-bit kernels?

hwc->config_base is an unsigned long, whereas CM3 qualifiers are 64-bit
values (and events such as CM3_PMU_EV_CM_REQ, CM3_PMU_EV_IO_REQ, and
CM3_PMU_EV_MEM_REQ require bits in 63:32 to be set in order to match any
requests).  Storing the qualifier in hwc->config1 (which is u64 in
struct hw_perf_event) preserves all 64 bits on CONFIG_32BIT.

[ ... ]

> +static void cm3_pmu_unregister(struct cm3_pmu *cm3_pmu)
> +{
> +	perf_pmu_unregister(&cm3_pmu->pmu);
> +	cpuhp_state_remove_instance(cm3_pmu_cpuhp_state, &cm3_pmu->node);
> +	hrtimer_cancel(&cm3_pmu->hrtimer);
> +	if (cm3_pmu->initialized && cm3_pmu_cluster_powered(cm3_pmu)) {
> +		cm3_pmu->irq = 0;
> +		cm3_pmu_hw_reset(cm3_pmu);
> +	}
> +}
> +
> +static int cm3_pmu_register(struct device *dev, struct cm3_pmu *cm3_pmu)
> +{

[ ... ]

> +	ret = perf_pmu_register(&cm3_pmu->pmu, name, -1);
> +	if (ret) {
> +		cpuhp_state_remove_instance(cm3_pmu_cpuhp_state, &cm3_pmu->node);
> +		return dev_err_probe(dev, ret, "failed to register %s\n", name);
> +	}

[Severity: Medium]
Should cm3_pmu_unregister() and the error path in cm3_pmu_register() use
cpuhp_state_remove_instance_nocalls() instead of
cpuhp_state_remove_instance()?

cpuhp_state_remove_instance() invokes cm3_pmu_offline_cpu() for each
online CPU while all CPUs remain in cpu_online_mask.  When more than one
CPU is online, cpumask_any_but(cpu_online_mask, cpu) finds a target CPU
and calls perf_pmu_migrate_context() on a PMU that has already been
unregistered (or failed perf_pmu_register()).

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261108-i6500-cm3-pmu-v1-0-bd119584891a@mobileye.com?part=4