* [RFC PATCH v2 01/15] misc/arm-cla: Add driver skeleton and documentation
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 02/15] misc/arm-cla: Add launch operation helpers Ryan Roberts
` (13 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Add the initial Kconfig and build-system plumbing for the Arm Core Local
Accelerator driver.
Introduce the common driver header and register definitions used by
later CLA support. The definitions cover the CLA MMIO frame, launch
response and status fields, standard accelerator registers, launch
opcodes, error codes and memory translation context state.
Add documentation describing the CLA programming model, its CPU-local
MMIO access rules, userspace assignment model, domain grouping and
expected boot state.
Co-developed-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
Documentation/misc-devices/arm-cla.rst | 217 +++++++++++++++++++++++++
Documentation/misc-devices/index.rst | 1 +
MAINTAINERS | 7 +
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/arm-cla/Kconfig | 10 ++
drivers/misc/arm-cla/Makefile | 1 +
drivers/misc/arm-cla/arm-cla-regs.h | 178 ++++++++++++++++++++
drivers/misc/arm-cla/arm-cla.h | 36 ++++
9 files changed, 452 insertions(+)
create mode 100644 Documentation/misc-devices/arm-cla.rst
create mode 100644 drivers/misc/arm-cla/Kconfig
create mode 100644 drivers/misc/arm-cla/Makefile
create mode 100644 drivers/misc/arm-cla/arm-cla-regs.h
create mode 100644 drivers/misc/arm-cla/arm-cla.h
diff --git a/Documentation/misc-devices/arm-cla.rst b/Documentation/misc-devices/arm-cla.rst
new file mode 100644
index 000000000000..6ce38b9b7e61
--- /dev/null
+++ b/Documentation/misc-devices/arm-cla.rst
@@ -0,0 +1,217 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================================
+Arm Core Local Accelerator (CLA)
+================================
+
+Arm CLA is an interface local to a CPU for programming accelerators that access
+memory via the CPU's MMU::
+
+ ┌───────┬───────┬───────┐ ┌───────┐
+ │ CPU │ MMU │ CLA │ │ Accel │─┐
+ │ │ │ │ │ │ │
+ │ ---MMIO--> <--> │ │
+ │ │ │ │ │ │ │
+ │ │ ,-----DMA-----> │ │
+ └───────┴────|──┴───────┘ └───────┘ │
+ ┌──────────v────────┐ └───────┘
+ │ Caches and memory │
+ └───────────────────┘
+
+
+Hardware
+========
+
+The CLA supports up to 8 attached accelerators, which are accessed by
+programming the CLA's MMIO registers. Operations are launched to an accelerator
+and are polled for completion. CLA does not raise interrupts.
+
+::
+
+ CPU CLA Accel
+ |--- write DATA[7:0] -->| |
+ |--- write LAUNCH ----->|---- launch ---->|
+ |<--- poll LRESP -------| |
+ | | |
+ |<--- poll STATUS ------|<--- complete ---|
+
+Each operation can take a 512-bit payload in the DATA registers. After handling
+a LAUNCH write, CLA indicates the launch status in the LRESP register. A further
+operation can only be launched after LRESP indicates completion of the previous
+launch.
+
+Some operations continue to run asynchronously on the accelerator after launch
+completion. In this case progress is tracked by polling the STATUS register.
+When the CLA updates the STATUS register, it also raises an event which will
+wake an in-progress WFE (wait for event) instruction on the local CPU.
+
+The CLA's MMIO registers are not accessible from remote CPUs. Although each CLA
+has a unique physical address, accesses from remote CPUs are read as zero and
+write ignored.
+
+The CLA registers are accessible from four different Privilege Level (PL)
+frames, with usage inteded to map to EL0 - EL3. The PLxCTRL registers may be
+written via a higher PL frame to suppress access to accelerators via a lower PL
+frame.
+
+The CPU and CLA share an MMU, although FEAT_TTCNP (common not private) is
+implemented, allowing both CPU and CLA to independently opt into and out of
+sharing TLB entries at runtime. TLB invalidation is performed via the CPU TLBI
+instructions; any TLBI instruction that targets the CLA's local CPU will also
+implicitly target the CLA.
+
+CLA has its own set of Memory Translation Context (MTC) registers, distinct from
+the CPU. A PL can set the MTC registers corresponding to an equivalent CPU
+Exception Level (EL) (eg. TCR_EL2 configurable only from the PL2 and PL3 MMIO
+frame).
+
+Faults during address translation are reported by the accelerator in its
+registers and in STATUS. While polling for work completion, software fixes up
+the faults and notifies the accelerator with RESOLVE operations.
+
+Accesses to the MMIO registers must be aligned 64-bit loads and stores, and the
+registers are mapped with Device-nGnRE attribute. Invalid accesses (unaligned,
+atomic, badly sized, etc) to the MMIO frame are either read as zero and write
+ignored, or cause a data abort, depending on the platform. Invalid access will
+never cause an SError.
+
+
+Inter-Accelerator Communication
+-------------------------------
+
+On some platforms, multiple accelerators, each attached to a separate CLA within
+a cluster, are also directly connected to each other via a shared bus to
+accelerate cooperation between accelerators. The accelerators sharing a bus
+cannot be isolated from each other. When collaborative operations are launched
+on each of the participating accelerators, they synchronize over the bus,
+stalling until all are ready.
+
+
+Intended SW Usage Model
+=======================
+
+CLA is designed for its PL0 MMIO frame to be mapped into user space and for user
+space to directly launch accelerator operations and poll for completion. It has
+been observed that for some use cases, the operation execution time is small and
+a trip through the kernel would consume a significant amount of the CPU budget
+for preparing the next operation leading to a significant reduction in bandwidth
+through the accelerator.
+
+User space software is expected to create a thread to drive each CLA it is
+using, and for each thread to be pinned to the CLA's local CPU.
+
+Software should rely on WFE (wait for event) to reduce power consumption when
+polling STATUS.
+
+The CLA is intended to be configured, by privileged software via PL1 and/or PL2,
+so that it shares virtual addresses with the process to which it is assigned
+(SVA). In practice this means configuring the CLA's MTC to point to the same
+page tables and use the same ASID (and VMID if relevant) as the process to which
+it is assigned. This ensures the architectural TLB invalidations also correctly
+target the CLA's TLB entries.
+
+We investigated the possibility of having the CLA driver allocate private page
+tables, private ASIDs/VMIDs and implement an MMU notifier for invalidation, but
+that suffers from 2 issues; there is a possibility of over-invalidation since
+the ASID and VMID spaces overlap with the CPU's (minor), and when issuing a
+TLBI, VMID is implicitly taken from CPU's VTTBR_EL2.VMID, which won't match the
+CLA's private VMID - therefore, for a virtualization scenario, TLB invalidation
+becomes impossible (major).
+
+Because the CLA has its own MTC registers, it is correct and safe for it to be
+executing an operation on behalf of user process A, while its local CPU is
+executing a thread from user process B.
+
+
+Difficulties for software to deal with
+--------------------------------------
+
+Although each CLA is attached to a single CPU, not all CPUs have a CLA, and CLAs
+may have a different set of accelerators attached. Users need to probe around to
+find a suitable accelerator and bind their process to it.
+
+Since remote CPUs can't access the CLA, dealing with CPU hotplug migrating tasks
+is challenging. And virtualization breaks down if the hypervisor cannot
+guarantee that a vCPU is pinned to a CPU.
+
+Saving and restoring the internal state of the accelerator is an optional
+feature. If an accelerator cannot preserve in-flight work, preempting it cancels
+that work. Software must carefully consider how to balance forward-progress
+guarantees with preemption latency.
+
+
+Driver
+======
+
+Booting
+-------
+
+A host expects to be booted with CLAs in the following state:
+
+- All attached accelerators have STATUS_IDLE set.
+- PL2CTRL: AVAIL all enabled for attached accelerators.
+ DBG at the firmware's discretion, preferably all enabled.
+- TSCTRLOWNER.PL, TSOFFOWNER.PL, PMUOWNER.PL all EL2.
+
+A guest expects to be booted with CLAs in the following state:
+
+- All attached accelerators have STATUS_IDLE set.
+- PL1CTRL: AVAIL all enabled for attached accelerators.
+ DBG at the hypervisor's discretion, preferably all enabled.
+
+Linux discovers the base address of each CLA device in the firmware tables and
+creates platform devices. cla_dev behaves mostly like a regular platform device,
+but it can only be accessed from one specific CPU. CPU hotplug notifiers probe
+and teardown the CLA device.
+
+CLAs may be grouped into domains. The topology is described in the firmware
+tables, and the driver creates cla_domain objects containing one or more
+cla_dev.
+
+
+Assignment to userspace
+-----------------------
+
+One char device per accelerator class provides enumeration and mmap abilities
+through ``/dev/cla/<class-id>``. Accelerators belong to the same class when
+their IIDR.IMPLEMENTER and IIDR.PRODUCTID register values match. ``<class-id>``
+is encoded as ``<IIDR.IMPLEMENTER:IIDR.PRODUCTID>``, each part using 3 hex
+digits. All class files enumerate all physical CLAs, with the same device IDs,
+domain IDs and mmap offsets. ``ARM_CLA_PARAM_DEV_ACCELS`` reports only the
+accelerator slots belonging to the opened class, and may be zero.
+
+The user task queries the driver to find a suitable set of CLAs, and mmaps their
+registers. For each mapped domain, the driver creates a cla_ctx context. When
+the user accesses the registers, the driver's fault handler queues the cla_ctx
+and waits for assignment. A context reassignment switches the whole cla_domain,
+by calling each CPU in the domain to switch the CLA context. A time slice is
+given to each context before being deassigned.
+
+In order to access the registers, the task must be bound to the CLA's CPU with
+sched_setaffinity(). Accesses from remote CPUs are ignored. If the CPU gets
+offlined, the task is migrated to an online CPU, and the driver disables the
+offlined CLA. User space may choose to use a mechanism such as restartable
+sequences to be notified when its task has been migrated away from its intended
+CLA.
+
+The accelerators access memory using the CPU's MMU. When assigning a context,
+the driver sets up the CLA's Memory Translation Context (MTC) with the page
+directory (TTBR), address space ID (ASID) and configuration (TCR) of the
+context's mm. When receiving translation faults from the accelerator, the user
+space task accesses the faulting address from the CPU to trigger the fault to be
+fixed up by the kernel before launching a RESOLVE operation to the accelerator.
+
+
+Power management
+----------------
+
+When a CPU enters a deep low-power state, then depending on the platform, the
+attached CLA and accelerators might not retain their state. In that case the
+firmware is expected to save the CLA and accelerator states before entering CPU
+low-power state, and restore them after exiting. The CLA driver does not expect
+to notice a CPU deep idle. However, if an accelerator does not support context
+saving, userspace will notice from the STATUS register that its work was
+canceled during deep idle. Given that userspace would be polling STATUS, the CPU
+is unlikely to enter deep idle while the CLA is running. To ensure
+forward-progress the admin can disable deep idle states (see
+Documentation/admin-guide/pm/cpuidle.rst).
diff --git a/Documentation/misc-devices/index.rst b/Documentation/misc-devices/index.rst
index f911edaecbfa..419c8f5c04ef 100644
--- a/Documentation/misc-devices/index.rst
+++ b/Documentation/misc-devices/index.rst
@@ -13,6 +13,7 @@ fit into other categories.
ad525x_dpot
amd-sbi
+ arm-cla
bh1770glc
c2port
dw-xdata-pcie
diff --git a/MAINTAINERS b/MAINTAINERS
index 72294ddfa5b7..735f6527f920 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2209,6 +2209,13 @@ F: arch/arm64/include/asm/arch_timer.h
F: drivers/clocksource/arm_arch_timer.c
F: drivers/clocksource/arm_arch_timer_mmio.c
+ARM CORE LOCAL ACCELERATOR (CLA) DRIVER
+M: Ryan Roberts <ryan.roberts@arm.com>
+L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
+S: Supported
+F: Documentation/misc-devices/arm-cla.rst
+F: drivers/misc/arm-cla/
+
ARM ETHOS-U NPU DRIVER
M: Rob Herring (Arm) <robh@kernel.org>
M: Tomeu Vizoso <tomeu@tomeuvizoso.net>
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index 7364931dad3a..1f50c9d39b4c 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -586,4 +586,5 @@ source "drivers/misc/keba/Kconfig"
source "drivers/misc/amd-sbi/Kconfig"
source "drivers/misc/rp1/Kconfig"
source "drivers/misc/issei/Kconfig"
+source "drivers/misc/arm-cla/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index e8d8d5d88c0d..2615190fee17 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -71,3 +71,4 @@ obj-y += keba/
obj-y += amd-sbi/
obj-$(CONFIG_MISC_RP1) += rp1/
obj-$(CONFIG_INTEL_SSEI) += issei/
+obj-$(CONFIG_ARM_CLA) += arm-cla/
diff --git a/drivers/misc/arm-cla/Kconfig b/drivers/misc/arm-cla/Kconfig
new file mode 100644
index 000000000000..f8436a12c1e6
--- /dev/null
+++ b/drivers/misc/arm-cla/Kconfig
@@ -0,0 +1,10 @@
+config ARM_CLA
+ tristate "Arm Core Local Accelerator"
+ default n
+ depends on ARM64
+ help
+ Arm Core Local Accelerator (CLA) is coupled with a specific CPU and
+ provides an interface to enumerate and access attached hardware
+ accelerators. It is only accessible from its associated CPU.
+
+ If unsure, say N.
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
new file mode 100644
index 000000000000..a4e40e534e6a
--- /dev/null
+++ b/drivers/misc/arm-cla/Makefile
@@ -0,0 +1 @@
+# SPDX-License-Identifier: GPL-2.0-only
diff --git a/drivers/misc/arm-cla/arm-cla-regs.h b/drivers/misc/arm-cla/arm-cla-regs.h
new file mode 100644
index 000000000000..d20a88780032
--- /dev/null
+++ b/drivers/misc/arm-cla/arm-cla-regs.h
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Arm CLA driver - register definitions
+ *
+ * Copyright 2026 Arm Limited.
+ */
+#ifndef _ARM_CLA_REGS_H_
+#define _ARM_CLA_REGS_H_
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/types.h>
+
+/* Registers */
+#define CLA_REG_DATA(i) (0x00 + (8 * (i)))
+#define CLA_REG_LAUNCH 0x40
+#define CLA_REG_LRESP 0x48
+#define CLA_REG_PL0CTRL 0x60
+#define CLA_REG_PL1CTRL 0x68
+#define CLA_REG_PL2CTRL 0x70
+#define CLA_REG_EVENT 0x78
+#define CLA_REG_STATUS(i) (0x80 + (8 * (i)))
+#define CLA_REG_CLAAIDR 0xC0
+
+#define CLA_LAUNCH_OP GENMASK(3, 0)
+#define CLA_LAUNCH_NDATA_M1 GENMASK(6, 4)
+#define CLA_LAUNCH_SEQ BIT(7)
+#define CLA_LAUNCH_ACCID GENMASK(10, 8)
+#define CLA_LAUNCH_REGIDX GENMASK(63, 32)
+
+#define CLA_LRESP_PENDING BIT(0)
+#define CLA_LRESP_CODE GENMASK(2, 1)
+#define CLA_LRESP_ERRCODE GENMASK(7, 3)
+#define CLA_LRESP_DATANZ BIT(15)
+
+#define CLA_PLxCTRL_AVAIL BIT(0)
+#define CLA_PLxCTRL_DBGPERM GENMASK(3, 1)
+#define CLA_PLxCTRL_PREP(accid, v) ((u64)(v) << (8 * (accid)))
+
+#define CLA_STATUS_AVAIL BIT(0)
+#define CLA_STATUS_DMB BIT(1)
+#define CLA_STATUS_EABORT BIT(2)
+#define CLA_STATUS_IDLE BIT(4)
+#define CLA_STATUS_READY BIT(5)
+#define CLA_STATUS_FAULT BIT(6)
+#define CLA_STATUS_EXCEPT BIT(7)
+#define CLA_STATUS_SRMODE BIT(8)
+#define CLA_STATUS_USER GENMASK(63, 16)
+
+/* Some useful values for sanity checks */
+#define CLA_STATUS_STATE_IDLE (CLA_STATUS_AVAIL | \
+ CLA_STATUS_IDLE | \
+ CLA_STATUS_READY)
+#define CLA_STATUS_STATE_SRMODE (CLA_STATUS_AVAIL | \
+ CLA_STATUS_IDLE | \
+ CLA_STATUS_READY | \
+ CLA_STATUS_SRMODE)
+/* Bits we care about when checking the state */
+#define CLA_STATUS_STATE_MASK (CLA_STATUS_AVAIL | \
+ CLA_STATUS_EABORT | \
+ CLA_STATUS_IDLE | \
+ CLA_STATUS_READY | \
+ CLA_STATUS_FAULT | \
+ CLA_STATUS_EXCEPT | \
+ CLA_STATUS_SRMODE)
+
+/* Standard accelerator registers */
+#define CLA_REG_IIDR 0x0000
+#define CLA_REG_DEVARCH 0x0001
+#define CLA_REG_REVIDR 0x0002
+#define CLA_REG_IASSIZE 0x0003
+#define CLA_REG_ACAP 0x0004
+#define CLA_REG_FSARV 0x001f
+#define CLA_REG_FSAR(n) (0x0020 + (n))
+#define CLA_REG_TSCTRLOWNER 0x00c0
+#define CLA_REG_TSCTRL 0x00c8
+#define CLA_REG_TSOFFOWNER 0x00d0
+#define CLA_REG_TSVOFF 0x00d8
+#define CLA_REG_TSPOFF 0x00d9
+#define CLA_REG_PMUOWNER 0x0100
+#define CLA_REG_PMURESET 0x0108
+#define CLA_REG_PMUCTRL 0x0110
+#define CLA_REG_PMUSNAP 0x0111
+#define CLA_REG_PMUEVT(n) (0x0120 + (n))
+#define CLA_REG_PMUCNT(n) (0x0140 + (n))
+#define CLA_REG_PMUSCNT(n) (0x0160 + (n))
+#define CLA_REG_IASn 0x8000
+
+#define CLA_IIDR_PRODUCTID GENMASK(31, 20)
+#define CLA_IIDR_VARIANT GENMASK(19, 16)
+#define CLA_IIDR_REVISION GENMASK(15, 12)
+#define CLA_IIDR_IMPLEMENTER GENMASK(11, 0)
+
+#define CLA_DEVARCH_ARCHITECT GENMASK(31, 21)
+#define CLA_DEVARCH_PRESENT BIT(20)
+#define CLA_DEVARCH_REVISION GENMASK(19, 16)
+#define CLA_DEVARCH_ARCHID GENMASK(15, 0)
+
+#define CLA_REVIDR_REVISION GENMASK(31, 0)
+
+#define CLA_ACAP_SROP BIT(0)
+#define CLA_ACAP_REGSTATE BIT(1)
+#define CLA_ACAP_PMUCNTS GENMASK(4, 2)
+#define CLA_ACAP_TS BIT(5)
+
+#define CLA_FSAR_READ BIT(0)
+#define CLA_FSAR_WRITE BIT(1)
+#define CLA_FSAR_ADDR GENMASK(63, 6)
+
+#define CLA_TSCTRLOWNER_PL GENMASK(1, 0)
+#define CLA_TSCTRL_TS GENMASK(1, 0)
+#define CLA_TSOFFOWNER_PL GENMASK(1, 0)
+
+#define CLA_TSCTRL_ZERO 0
+#define CLA_TSCTRL_VIRTUAL 1
+#define CLA_TSCTRL_GUESTPHYSICAL 2
+#define CLA_TSCTRL_PHYSICAL 3
+
+#define CLA_PMUCTRL_EN BIT(0)
+#define CLA_PMUOWNER_PL GENMASK(1, 0)
+
+/* LAUNCH operations */
+#define CLA_LAUNCH_OP_RESET 0
+#define CLA_LAUNCH_OP_CMD 1
+#define CLA_LAUNCH_OP_CMDNR 2
+#define CLA_LAUNCH_OP_ENTERSR 4
+#define CLA_LAUNCH_OP_EXITSR 5
+#define CLA_LAUNCH_OP_SAVE 6
+#define CLA_LAUNCH_OP_RESTORE 7
+#define CLA_LAUNCH_OP_RESOLVE 9
+#define CLA_LAUNCH_OP_REGREAD 10
+#define CLA_LAUNCH_OP_REGWRITE 11
+#define CLA_LAUNCH_OP_SETCTX 12
+#define CLA_LAUNCH_OP_GETCTX 13
+
+/* Return codes */
+#define CLA_LRESP_OK 0
+#define CLA_LRESP_UNAVAIL 1
+#define CLA_LRESP_BUSY 2
+#define CLA_LRESP_ERROR 3
+
+#define CLA_ERRCODE_CSINT 0
+#define CLA_ERRCODE_CSOF 1
+#define CLA_ERRCODE_NOTIDLE 2
+#define CLA_ERRCODE_PERM 3
+#define CLA_ERRCODE_NOACC 4
+#define CLA_ERRCODE_INVAL 5
+#define CLA_ERRCODE_RESET 6
+
+/* Memory translation context */
+#define CLA_MTC_REGIDX_PL1 0
+#define CLA_MTC_REGIDX_PL2 64
+#define CLA_MTC_PL_SIZE 64
+
+/* Common register offsets */
+#define CLA_MTC_PSTATE 0
+#define CLA_MTC_TTBR0 1
+#define CLA_MTC_TTBR1 2
+#define CLA_MTC_TCR 3
+#define CLA_MTC_SCTLR 4
+#define CLA_MTC_MAIR 5
+#define CLA_MTC_TCR2 8
+
+/* EL2 specific register offsets */
+#define CLA_MTC_HCR_EL2 80
+#define CLA_MTC_VTTBR_EL2 81
+#define CLA_MTC_VTCR_EL2 82
+
+#define CLA_MTC_PSTATE_EL GENMASK(1, 0)
+#define CLA_MTC_PSTATE_PAN BIT(2)
+
+#define CLA_SRSTATE_0_SROP BIT(0)
+#define CLA_SRSTATE_0_REGSTATE GENMASK(15, 1)
+#define CLA_SRSTATE_1_STATUS GENMASK(63, 0)
+#define CLA_SRSTATE_2_SRACTIVE GENMASK(1, 0)
+#define CLA_SRSTATE_2_ADDR_MASK GENMASK(63, 6)
+
+#endif /* _ARM_CLA_REGS_H_ */
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
new file mode 100644
index 000000000000..1e8ffca28bc6
--- /dev/null
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Arm CLA driver - internal definitions
+ *
+ * Copyright 2026 Arm Limited.
+ */
+#ifndef _ARM_CLA_H_
+#define _ARM_CLA_H_
+
+#include "arm-cla-regs.h"
+
+/* Number of accelerators per CLA */
+#define CLA_NUM_ACC 8
+#define CLA_NUM_DATA_REGS 8
+#define CLA_SRSTATE_LEN 8
+
+/**
+ * struct cla_dev - CLA device
+ *
+ * Immutable state:
+ * @cpu: The CPU this CLA is attached to.
+ * @dev: The platform device.
+ */
+struct cla_dev {
+ unsigned int cpu;
+ struct device *dev;
+};
+
+#define cla_dbg(_dev, fmt, ...) \
+ dev_dbg((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
+#define cla_info(_dev, fmt, ...) \
+ dev_info((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
+#define cla_err(_dev, fmt, ...) \
+ dev_err((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
+
+#endif /* _ARM_CLA_H_ */
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 02/15] misc/arm-cla: Add launch operation helpers
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 01/15] misc/arm-cla: Add driver skeleton and documentation Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 16:17 ` Greg Kroah-Hartman
2026-10-15 14:25 ` [RFC PATCH v2 03/15] dt-bindings: misc: Add Arm Core Local Accelerator Ryan Roberts
` (12 subsequent siblings)
14 siblings, 1 reply; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
CLA commands are issued by writing optional payload registers,
programming the LAUNCH register and polling LRESP until the hardware
accepts or rejects the operation.
Add a common launch helper that performs this sequence on the CLA's
local CPU, waits for LRESP completion and translates launch response
codes into Linux errors.
Build accelerator reset and register read and write support on top of
it. The register read and write helpers split larger accesses into
multiple launch operations when an access crosses an eight-register
window.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Makefile | 6 +
drivers/misc/arm-cla/arm-cla.h | 24 +++
drivers/misc/arm-cla/cla-init.c | 12 ++
drivers/misc/arm-cla/cla-ops.c | 293 ++++++++++++++++++++++++++++++++
4 files changed, 335 insertions(+)
create mode 100644 drivers/misc/arm-cla/cla-init.c
create mode 100644 drivers/misc/arm-cla/cla-ops.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index a4e40e534e6a..3fa8567c8b3e 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -1 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
+
+arm-cla-y := \
+ cla-init.o \
+ cla-ops.o
+
+obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 1e8ffca28bc6..4884e4804ab3 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -7,6 +7,9 @@
#ifndef _ARM_CLA_H_
#define _ARM_CLA_H_
+#include <linux/device.h>
+#include <linux/io.h>
+
#include "arm-cla-regs.h"
/* Number of accelerators per CLA */
@@ -19,10 +22,12 @@
*
* Immutable state:
* @cpu: The CPU this CLA is attached to.
+ * @regs: Registers accessed by the kernel.
* @dev: The platform device.
*/
struct cla_dev {
unsigned int cpu;
+ void __iomem *regs;
struct device *dev;
};
@@ -33,4 +38,23 @@ struct cla_dev {
#define cla_err(_dev, fmt, ...) \
dev_err((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
+static inline u64 cla_reg_read(struct cla_dev *dev, off_t reg)
+{
+ /* MMIO reads are all relaxed, since the driver doesn't use DMA. */
+ return readq_relaxed(dev->regs + reg);
+}
+
+static inline void cla_reg_write(struct cla_dev *dev, off_t reg, u64 val)
+{
+ /* MMIO writes are all relaxed, since the driver doesn't use DMA. */
+ return writeq_relaxed(val, dev->regs + reg);
+}
+
+int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
+int cla_op_reset(struct cla_dev *dev, unsigned int accid);
+int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
+ size_t nregs, u64 *regs);
+int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
+ unsigned int regidx, size_t nregs, u64 *regs);
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
new file mode 100644
index 000000000000..3d1f47592842
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - probing and initialization
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/module.h>
+
+MODULE_DESCRIPTION("Arm Core Local Accelerator");
+MODULE_AUTHOR("Arm Limited");
+MODULE_LICENSE("GPL");
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
new file mode 100644
index 000000000000..46026304a057
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -0,0 +1,293 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - Launch operations
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/iopoll.h>
+
+#include "arm-cla.h"
+
+/* Time to wait between two LRESP reads */
+#define CLA_LRESP_DELAY_US 1
+
+/*
+ * Time to wait for LRESP_PENDING to clear. Commands like REGREAD should
+ * complete in a few cycles, but ENTERSR and RESET may need to clean up
+ * some very large states depending on the work interrupted, and may need
+ * 1us or more. Some software models need a lot more time.
+ */
+#define CLA_LRESP_TIMEOUT_US 1000000 /* 1s! */
+
+/* Time to wait between two STATUS reads when polling for DMB */
+#define CLA_STATUS_DMB_DELAY_US 10
+
+/* Time to wait for STATUS_DMB to be set after a successful RESET or ENTERSR. */
+#define CLA_STATUS_DMB_TIMEOUT_US 1000000 /* 1s! */
+
+enum cla_launch_data_mode {
+ CLA_DATA_NONE,
+ CLA_DATA_IN,
+ CLA_DATA_OUT,
+};
+
+struct cla_launch {
+ u8 op; /* opcode */
+ u8 ndata_m1; /* Data size minus 1 */
+ u8 accid; /* Accelerator ID */
+ bool seq; /* part of compound cmd */
+ u32 regidx; /* Register index */
+
+ enum cla_launch_data_mode data_mode;
+ u64 *data; /* In/out data */
+
+ u8 errcode; /* Output error code */
+};
+
+/**
+ * cla_op_launch - Launch operation.
+ * @dev: CLA device.
+ * @launch: LAUNCH settings.
+ *
+ * 1. Check that no previous operation is pending.
+ * 2. If data_mode is %CLA_DATA_IN, write DATA registers.
+ * 3. Launch the operation, and wait for the response.
+ * 4. If data_mode is %CLA_DATA_OUT, read DATA registers into @launch->data.
+ *
+ * Return:
+ * * %0 - Success.
+ * * %-EAGAIN - A previous operation is still pending. Nothing was
+ * written to DATA or LAUNCH.
+ * * %-ETIMEDOUT - LAUNCH timed out (possibly no CLA at this address).
+ * Unless LRESP_PENDING eventually clears, this is
+ * unrecoverable.
+ * * %-ENODEV - Accelerator not available.
+ * * %-EBUSY - Accelerator is busy.
+ * * %-EIO - LAUNCH error. @launch->errcode contains the LRESP
+ * error code.
+ */
+static int cla_op_launch(struct cla_dev *dev, struct cla_launch *launch)
+{
+ int i;
+ int ret;
+ u64 lresp;
+
+ if (WARN_ON(smp_processor_id() != dev->cpu))
+ return -EINVAL;
+
+ lresp = cla_reg_read(dev, CLA_REG_LRESP);
+ if (FIELD_GET(CLA_LRESP_PENDING, lresp))
+ return -EAGAIN;
+
+ if (launch->data_mode == CLA_DATA_IN)
+ for (i = 0; i < launch->ndata_m1 + 1; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), launch->data[i]);
+
+ /*
+ * For operations that return a response, LAUNCH sets LRESP_PENDING or
+ * provides an immediate response. So a previously completed error
+ * response does not block a new LAUNCH and does not need to be
+ * explicitly cleared.
+ */
+ cla_reg_write(dev, CLA_REG_LAUNCH,
+ FIELD_PREP(CLA_LAUNCH_OP, launch->op) |
+ FIELD_PREP(CLA_LAUNCH_NDATA_M1, launch->ndata_m1) |
+ FIELD_PREP(CLA_LAUNCH_ACCID, launch->accid) |
+ FIELD_PREP(CLA_LAUNCH_SEQ, launch->seq) |
+ FIELD_PREP(CLA_LAUNCH_REGIDX, launch->regidx));
+
+ ret = cla_op_wait_lresp(dev, &lresp);
+ if (ret) {
+ cla_err(dev, "launch failed with %d\n", ret);
+ return ret;
+ }
+
+ switch (FIELD_GET(CLA_LRESP_CODE, lresp)) {
+ case CLA_LRESP_OK:
+ break;
+ case CLA_LRESP_UNAVAIL:
+ return -ENODEV;
+ case CLA_LRESP_BUSY:
+ return -EBUSY;
+ case CLA_LRESP_ERROR:
+ launch->errcode = FIELD_GET(CLA_LRESP_ERRCODE, lresp);
+ return -EIO;
+ }
+
+ if (launch->data_mode == CLA_DATA_OUT)
+ for (i = 0; i < launch->ndata_m1 + 1; i++)
+ launch->data[i] = cla_reg_read(dev, CLA_REG_DATA(i));
+
+ return 0;
+}
+
+/**
+ * cla_op_wait_lresp - Wait for any LAUNCH op to complete.
+ * @dev: CLA device.
+ * @lresp: last LRESP value read.
+ *
+ * Return: 0 on success, -ETIMEDOUT in case of timeout.
+ */
+int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp)
+{
+ return readq_relaxed_poll_timeout_atomic(dev->regs + CLA_REG_LRESP,
+ *lresp, FIELD_GET(CLA_LRESP_PENDING, *lresp) == 0,
+ CLA_LRESP_DELAY_US, CLA_LRESP_TIMEOUT_US);
+}
+
+/*
+ * Wait for STATUS_DMB to be 1. Although RESET and ENTERSR both wait for
+ * accelerators to return to STATUS_IDLE, some writes may still be pending. When
+ * all accesses reach the PoC, STATUS_DMB is 1.
+ *
+ * This is needed for a few reasons:
+ *
+ * - Propagation of STATUS_EABORT. Until all writes are flushed and STATUS_DMB
+ * is 1, external aborts may still happen. The next context must not see this
+ * external abort reported in its status.
+ *
+ * Note that EABORT cannot be injected into the previous context, it is not
+ * part of the saved and restored STATUS bits.
+ *
+ * - Turning CLA power off.
+ *
+ * - Migrating an accelerator owner across CPUs, although the driver does not
+ * support it.
+ *
+ * Return: -ETIMEDOUT on timeout. On success, return 0 and set *status to the
+ * last status read.
+ */
+static int cla_wait_status_dmb(struct cla_dev *dev, unsigned int accid, u64 *status)
+{
+ return poll_timeout_us(*status = cla_reg_read(dev, CLA_REG_STATUS(accid)),
+ *status & CLA_STATUS_DMB,
+ CLA_STATUS_DMB_DELAY_US,
+ CLA_STATUS_DMB_TIMEOUT_US,
+ /* sleep_before_read */ false);
+}
+
+/**
+ * cla_op_reset - Launch RESET operation for this accelerator.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ *
+ * Return: 0 on success, 1 if there is no accelerator with this ID, or an error.
+ */
+int cla_op_reset(struct cla_dev *dev, unsigned int accid)
+{
+ int ret;
+ u64 status;
+ struct cla_launch launch = {
+ .op = CLA_LAUNCH_OP_RESET,
+ .accid = accid,
+ };
+
+ ret = cla_op_launch(dev, &launch);
+ if (ret) {
+ if (ret == -EIO && launch.errcode == CLA_ERRCODE_NOACC)
+ return 1;
+ return ret;
+ }
+
+ /* Wait for outstanding writes */
+ ret = cla_wait_status_dmb(dev, accid, &status);
+ if (ret)
+ return ret;
+
+ /*
+ * If STATUS.EABORT is set, a RAS error report is available. Launch a
+ * second reset to clear the bit. No need to wait for STATUS.DMB again
+ * since all accelerators are idle from the first reset, so no more
+ * memory writes could have been generated.
+ */
+ if (status & CLA_STATUS_EABORT)
+ return cla_op_launch(dev, &launch);
+
+ return 0;
+}
+
+/* regidx is unsigned long here to avoid overflow during calculations */
+static int cla_op_access_reg(struct cla_dev *dev, u8 op,
+ enum cla_launch_data_mode data_mode,
+ unsigned int accid, unsigned long regidx,
+ size_t nregs, u64 *regs)
+{
+ int ret = -EINVAL;
+ unsigned long max_regidx;
+ struct cla_launch launch = {
+ .op = op,
+ .accid = accid,
+ .data_mode = data_mode,
+ };
+
+ switch (op) {
+ case CLA_LAUNCH_OP_REGREAD:
+ case CLA_LAUNCH_OP_REGWRITE:
+ max_regidx = 0x100000000;
+ break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ if (WARN_ON(regidx + nregs > max_regidx))
+ return -EINVAL;
+
+ /* 1 to 8 registers accessed at a time, within the same 8-reg group */
+ while (nregs > 0) {
+ u64 reg_group = ALIGN_DOWN(regidx, 8);
+ u64 max_reg = min(regidx + nregs, reg_group + 8);
+ u8 ndata = max_reg - regidx;
+
+ launch.ndata_m1 = ndata - 1;
+ launch.regidx = regidx;
+ launch.data = regs;
+
+ ret = cla_op_launch(dev, &launch);
+ if (ret)
+ break;
+
+ regidx += ndata;
+ regs += ndata;
+ nregs -= ndata;
+ }
+
+ return ret;
+}
+
+/**
+ * cla_op_regread - Launch REGREAD operations.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @regidx: first register index.
+ * @nregs: number of registers. Can be greater than 8 (accessed with multiple
+ * REGREAD operations).
+ * @regs: array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_regread(struct cla_dev *dev, unsigned int accid,
+ unsigned int regidx, size_t nregs, u64 *regs)
+{
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_REGREAD, CLA_DATA_OUT,
+ accid, regidx, nregs, regs);
+}
+
+/**
+ * cla_op_regwrite - Launch REGWRITE operations.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @regidx: first register index.
+ * @nregs: number of registers. Can be greater than 8 (accessed with multiple
+ * REGWRITE operations).
+ * @regs: array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
+ unsigned int regidx, size_t nregs, u64 *regs)
+{
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_REGWRITE, CLA_DATA_IN,
+ accid, regidx, nregs, regs);
+}
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC PATCH v2 02/15] misc/arm-cla: Add launch operation helpers
2026-10-15 14:25 ` [RFC PATCH v2 02/15] misc/arm-cla: Add launch operation helpers Ryan Roberts
@ 2026-10-15 16:17 ` Greg Kroah-Hartman
2026-10-15 16:32 ` Ryan Roberts
0 siblings, 1 reply; 20+ messages in thread
From: Greg Kroah-Hartman @ 2026-10-15 16:17 UTC (permalink / raw)
To: Ryan Roberts
Cc: Arnd Bergmann, Catalin Marinas, Will Deacon, Mark Rutland,
Marc Zyngier, Oliver Upton, Alex Williamson, Jason Gunthorpe,
Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet, linux-kernel,
linux-arm-kernel, kvmarm, kvm, devicetree, dri-devel, linux-doc
On Thu, Oct 15, 2026 at 03:25:36PM +0100, Ryan Roberts wrote:
> From: Jean-Philippe Brucker <jpb@kernel.org>
>
> CLA commands are issued by writing optional payload registers,
> programming the LAUNCH register and polling LRESP until the hardware
> accepts or rejects the operation.
>
> Add a common launch helper that performs this sequence on the CLA's
> local CPU, waits for LRESP completion and translates launch response
> codes into Linux errors.
>
> Build accelerator reset and register read and write support on top of
> it. The register read and write helpers split larger accesses into
> multiple launch operations when an access crosses an eight-register
> window.
>
> Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
You forgot to sign off on patches from others :(
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC PATCH v2 02/15] misc/arm-cla: Add launch operation helpers
2026-10-15 16:17 ` Greg Kroah-Hartman
@ 2026-10-15 16:32 ` Ryan Roberts
0 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 16:32 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Arnd Bergmann, Catalin Marinas, Will Deacon, Mark Rutland,
Marc Zyngier, Oliver Upton, Alex Williamson, Jason Gunthorpe,
Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet, linux-kernel,
linux-arm-kernel, kvmarm, kvm, devicetree, dri-devel, linux-doc
On 15/10/2026 17:17, Greg Kroah-Hartman wrote:
> On Thu, Oct 15, 2026 at 03:25:36PM +0100, Ryan Roberts wrote:
>> From: Jean-Philippe Brucker <jpb@kernel.org>
>>
>> CLA commands are issued by writing optional payload registers,
>> programming the LAUNCH register and polling LRESP until the hardware
>> accepts or rejects the operation.
>>
>> Add a common launch helper that performs this sequence on the CLA's
>> local CPU, waits for LRESP completion and translates launch response
>> codes into Linux errors.
>>
>> Build accelerator reset and register read and write support on top of
>> it. The register read and write helpers split larger accesses into
>> multiple launch operations when an access crosses an eight-register
>> window.
>>
>> Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
>
> You forgot to sign off on patches from others :(
Ahh sh*t, sorry about that. Hopefully given this is "just" an RFC that's not a
big deal? Else, let me know and I'll resend.
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC PATCH v2 03/15] dt-bindings: misc: Add Arm Core Local Accelerator
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 01/15] misc/arm-cla: Add driver skeleton and documentation Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 02/15] misc/arm-cla: Add launch operation helpers Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 04/15] misc/arm-cla: Probe firmware-described devices Ryan Roberts
` (11 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
Arm Core Local Accelerator (CLA) provides a CPU-local interface to
accelerators that access memory through the local CPU's MMU. CLAs whose
accelerators cannot be isolated from each other form an isolation
domain.
Add a binding for arm,cla describing the register range, CPU
association, DMA coherency and isolation domain. Document the
privilege-level register frames, including reduced ranges exposed to
virtual machines, and provide an example of two CLAs sharing a domain.
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
.../devicetree/bindings/misc/arm,cla.yaml | 77 +++++++++++++++++++
MAINTAINERS | 1 +
2 files changed, 78 insertions(+)
create mode 100644 Documentation/devicetree/bindings/misc/arm,cla.yaml
diff --git a/Documentation/devicetree/bindings/misc/arm,cla.yaml b/Documentation/devicetree/bindings/misc/arm,cla.yaml
new file mode 100644
index 000000000000..b86a2befd09a
--- /dev/null
+++ b/Documentation/devicetree/bindings/misc/arm,cla.yaml
@@ -0,0 +1,77 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/misc/arm,cla.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Arm Core Local Accelerator (CLA)
+
+maintainers:
+ - Ryan Roberts <ryan.roberts@arm.com>
+
+description:
+ A Core Local Accelerator (CLA) provides a CPU-local interface to up to eight
+ accelerators. The accelerators access memory through the local CPU's MMU. Each
+ CLA has its own register range, which can only be accessed by its local CPU.
+ Completion is polled through the registers; CLA does not generate interrupts.
+
+properties:
+ compatible:
+ const: arm,cla
+
+ reg:
+ maxItems: 1
+ description:
+ A single contiguous register range starting at the PL0 frame, aligned to
+ 256 KiB. Each privilege-level frame occupies 64 KiB, with PL0, PL1, PL2
+ and PL3 at offsets 0, 0x10000, 0x20000 and 0x30000 respectively. The
+ complete range is 256 KiB. Higher-privilege frames may be omitted when
+ unavailable, for example in a virtual machine. The range must include at
+ least PL0 and PL1 when running at EL1 and additionally PL2 when running at
+ EL2.
+
+ cpu:
+ $ref: /schemas/types.yaml#/definitions/phandle
+ description:
+ Phandle to the CPU for which this CLA is local. Only that CPU can access
+ the CLA's registers.
+
+ dma-coherent: true
+
+ arm,domain:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ Identifier of the CLA isolation domain. CLAs whose attached accelerators
+ cannot be isolated from each other, for example because they share an
+ inter-accelerator communication bus, must have the same domain ID.
+ Different isolation domains must have different IDs. IDs need not be
+ consecutive. A CLA that can be isolated from all other CLAs has its own
+ domain ID.
+
+required:
+ - compatible
+ - reg
+ - cpu
+ - dma-coherent
+ - arm,domain
+
+additionalProperties: false
+
+examples:
+ - |
+ /* Two CPU-local CLAs in the same isolation domain. */
+ cla@10000000 {
+ compatible = "arm,cla";
+ reg = <0x10000000 0x40000>;
+ cpu = <&cpu0>;
+ dma-coherent;
+ arm,domain = <0>;
+ };
+
+ cla@10040000 {
+ compatible = "arm,cla";
+ reg = <0x10040000 0x40000>;
+ cpu = <&cpu1>;
+ dma-coherent;
+ arm,domain = <0>;
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index 735f6527f920..98a25b3de9b4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2213,6 +2213,7 @@ ARM CORE LOCAL ACCELERATOR (CLA) DRIVER
M: Ryan Roberts <ryan.roberts@arm.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Supported
+F: Documentation/devicetree/bindings/misc/arm,cla.yaml
F: Documentation/misc-devices/arm-cla.rst
F: drivers/misc/arm-cla/
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 04/15] misc/arm-cla: Probe firmware-described devices
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (2 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 03/15] dt-bindings: misc: Add Arm Core Local Accelerator Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 05/15] misc/arm-cla: Initialize devices on CPU bringup Ryan Roberts
` (10 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Discover CLA devices from firmware and instantiate the driver with
platform_driver_probe(), since the set of devices is fixed at boot. Map
the MMIO frame, select the kernel-accessible privilege-level frame and
associate each CLA with the CPU named by firmware.
Add the topology code used to group CLAs into domains. Domains represent
the smallest unit that can be assigned independently, because CLAs in
the same domain cannot be isolated from each other. Build lookup tables
indexed by CPU and by mmap page offset for later fast access.
Register CPU hotplug callbacks so that hardware setup and teardown can
run on the CPU local to each CLA, which is the only CPU allowed to
access that CLA's registers.
Co-developed-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Makefile | 3 +-
drivers/misc/arm-cla/arm-cla.h | 46 ++++++
drivers/misc/arm-cla/cla-init.c | 222 ++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 172 +++++++++++++++++++++
4 files changed, 442 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/arm-cla/cla-topology.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index 3fa8567c8b3e..84ed1a054a21 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -2,6 +2,7 @@
arm-cla-y := \
cla-init.o \
- cla-ops.o
+ cla-ops.o \
+ cla-topology.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 4884e4804ab3..61176c4fce0f 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -17,6 +17,8 @@
#define CLA_NUM_DATA_REGS 8
#define CLA_SRSTATE_LEN 8
+struct cla_domain;
+
/**
* struct cla_dev - CLA device
*
@@ -24,13 +26,41 @@
* @cpu: The CPU this CLA is attached to.
* @regs: Registers accessed by the kernel.
* @dev: The platform device.
+ * @pfn: Page of registers assigned to user.
+ * @pg_offset: Mmap offset of this device.
+ * @domain: The domain this CLA belongs to.
*/
struct cla_dev {
unsigned int cpu;
void __iomem *regs;
struct device *dev;
+ unsigned long pfn;
+ unsigned long pg_offset;
+ struct cla_domain *domain;
+};
+
+/**
+ * struct cla_domain - Collection of cla_dev
+ *
+ * Immutable state:
+ * @id: Consecutive userspace domain identifier.
+ * @pg_offset: Mmap offset of the first device.
+ * @nr_devs: Number of devices in the domain.
+ * @devs: Devices.
+ */
+struct cla_domain {
+ unsigned int id;
+ unsigned long pg_offset;
+ unsigned int nr_devs;
+ struct cla_dev **devs;
};
+extern struct xarray cla_domains;
+extern unsigned int cla_nr_domains;
+extern struct cla_dev **cla_lut_cpu;
+extern struct cla_dev **cla_lut_pg;
+extern unsigned int cla_nr_devs;
+
#define cla_dbg(_dev, fmt, ...) \
dev_dbg((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
#define cla_info(_dev, fmt, ...) \
@@ -38,6 +68,13 @@ struct cla_dev {
#define cla_err(_dev, fmt, ...) \
dev_err((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
+#define CLA_REG_SIZE SZ_64K
+#define CLA_FRAME_SIZE (4 * CLA_REG_SIZE)
+
+/* Return the registers corresponding to this privilege level */
+#define cla_get_regs(base, pl) \
+ ((typeof(base))((uintptr_t)(base) + (pl) * CLA_REG_SIZE))
+
static inline u64 cla_reg_read(struct cla_dev *dev, off_t reg)
{
/* MMIO reads are all relaxed, since the driver doesn't use DMA. */
@@ -50,6 +87,15 @@ static inline void cla_reg_write(struct cla_dev *dev, off_t reg, u64 val)
return writeq_relaxed(val, dev->regs + reg);
}
+/*
+ * If we're at EL2, use PL2. If we're a guest or nVHE host, use PL1.
+ */
+#define cla_kernel_pl (is_kernel_in_hyp_mode() ? 2 : 1)
+
+struct cla_domain *cla_dev_domain_get(struct cla_dev *dev);
+int cla_domains_finalise(void);
+void cla_domains_free(void);
+
int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
int cla_op_reset(struct cla_dev *dev, unsigned int accid);
int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 3d1f47592842..6818fb8c38ed 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -5,7 +5,229 @@
* Copyright 2026 Arm Limited.
*/
+#include <linux/cpuhotplug.h>
+#include <linux/cpuhplock.h>
#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/smp.h>
+
+#include <asm/virt.h>
+
+#include "arm-cla.h"
+
+static int cla_cpuhp_state = -1;
+
+static int cla_dev_setup(unsigned int cpu)
+{
+ struct cla_dev *dev;
+
+ dev = cla_lut_cpu[cpu];
+ if (!dev)
+ return 0;
+
+ if (WARN_ON(smp_processor_id() != cpu || dev->cpu != cpu))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int cla_dev_teardown(unsigned int cpu)
+{
+ struct cla_dev *dev;
+
+ /*
+ * Careful what we return here, the teardown path isn't really allowed
+ * to fail (BUG_ON in kernel/cpu.c)
+ */
+ dev = cla_lut_cpu[cpu];
+ if (!dev)
+ return 0;
+
+ return 0;
+}
+
+static int cla_of_to_cpu(struct device_node *of_node)
+{
+ int cpu;
+ int ret;
+ u32 cpu_phandle;
+ struct device_node *cpu_node;
+
+ if (!of_node)
+ return -ENODEV;
+
+ ret = of_property_read_u32(of_node, "cpu", &cpu_phandle);
+ if (WARN_ON(ret))
+ return -EINVAL;
+
+ cpu_node = of_find_node_by_phandle(cpu_phandle);
+ if (WARN_ON(!cpu_node))
+ return -EINVAL;
+
+ cpu = of_cpu_node_to_id(cpu_node);
+ of_node_put(cpu_node);
+
+ return cpu;
+}
+
+static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
+ void __iomem *regs, phys_addr_t base)
+{
+ struct cla_dev *dev;
+
+ dev = devm_kzalloc(parent, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return ERR_PTR(-ENOMEM);
+
+ dev->pfn = __phys_to_pfn(base);
+ dev->regs = cla_get_regs(regs, cla_kernel_pl);
+ dev->cpu = cpu;
+ dev->dev = parent;
+
+ /* Attempt to find device domain, or allocate a new one */
+ dev->domain = cla_dev_domain_get(dev);
+ if (IS_ERR(dev->domain))
+ return ERR_CAST(dev->domain);
+
+ cla_nr_devs++;
+
+ return dev;
+}
+
+static int cla_probe(struct platform_device *pdev)
+{
+ int cpu;
+ void __iomem *reg;
+ size_t reg_size;
+ struct cla_dev *dev;
+ struct resource *res;
+
+ /*
+ * TODO: the firmware maps this as well to access PL3, and the guest
+ * will map PL1 and PL0. Avoid TLB attr mismatches by only mapping what
+ * we need.
+ */
+ reg = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
+ if (IS_ERR(reg)) {
+ dev_err(&pdev->dev, "could not map CLA registers\n");
+ return PTR_ERR(reg);
+ }
+
+ if (!IS_ALIGNED(res->start, SZ_256K)) {
+ dev_err(&pdev->dev, "invalid CLA registers alignment\n");
+ return -EINVAL;
+ }
+
+ /*
+ * As a guest we may not get PL3 or PL2. Tolerate CLAs smaller
+ * than 4*CLA_REG_SIZE.
+ */
+ reg_size = resource_size(res);
+ if (reg_size > CLA_FRAME_SIZE ||
+ reg_size < (cla_kernel_pl + 1) * CLA_REG_SIZE) {
+ dev_err(&pdev->dev, "unexpected CLA registers size\n");
+ return -EINVAL;
+ }
+
+ cpu = cla_of_to_cpu(pdev->dev.of_node);
+ if (cpu < 0)
+ return cpu;
+
+ /* Reject CLAs for offline CPUs. See cla_module_init(). */
+ if (!cpu_online(cpu))
+ return -ENODEV;
+
+ dev = cla_dev_alloc(&pdev->dev, cpu, reg, res->start);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
+
+ return 0;
+}
+
+static const struct of_device_id cla_of_match[] = {
+ {.compatible = "arm,cla",},
+ {},
+};
+MODULE_DEVICE_TABLE(of, cla_of_match);
+
+static struct platform_driver cla_driver = {
+ .driver = {
+ .name = "arm-cla",
+ .of_match_table = cla_of_match,
+ },
+};
+
+static int __init cla_module_init(void)
+{
+ bool driver_registered = false;
+ int ret;
+
+ /*
+ * For now, we reject any CLAs discovered via FW tables, whose CPU is
+ * offline during init. This keeps things simple by allowing us to probe
+ * all HW at init time. Keep the online CPU set stable so that the
+ * cpu_online() check in cla_probe() is consistent with the online cpus
+ * when we call cpuhp_setup_state().
+ */
+ scoped_guard(cpus_read_lock) {
+ /*
+ * CPUs may be hotplugged, but all CLAs are described by
+ * firmware so the probe can be synchronous. This only sets up
+ * the resources, and CPUHP callbacks do the actual peeking and
+ * poking. This returns an error when no CLA has an online CPU.
+ */
+ ret = platform_driver_probe(&cla_driver, cla_probe);
+ if (ret) {
+ if (ret != -ENODEV)
+ pr_err("arm-cla: probe failed with %d\n", ret);
+ /* Some domains may have been created during probe. */
+ goto err_hardware;
+ }
+ driver_registered = true;
+
+ ret = cla_domains_finalise();
+ if (ret) {
+ pr_err("arm-cla: failed to finalise domains: %d", ret);
+ goto err_hardware;
+ }
+
+ /*
+ * Each CPU initializes their own CLA. CPUHP uses a pair of
+ * smp_mb() when calling the startup callback, ensuring that
+ * cla_dev_setup() reads fully initialized cla_lut_cpu and
+ * cla_dev structures.
+ */
+ ret = cpuhp_setup_state_cpuslocked(CPUHP_AP_ONLINE_DYN,
+ "arm-cla", cla_dev_setup,
+ cla_dev_teardown);
+ if (ret < 0) {
+ pr_err("arm-cla: failed to setup cpuhp: %d", ret);
+ goto err_hardware;
+ }
+ cla_cpuhp_state = ret;
+ }
+
+ return 0;
+
+err_hardware:
+ if (cla_cpuhp_state >= 0)
+ cpuhp_remove_state(cla_cpuhp_state);
+ cla_domains_free();
+ if (driver_registered)
+ platform_driver_unregister(&cla_driver);
+ return ret;
+}
+
+static void __exit cla_module_exit(void)
+{
+ cpuhp_remove_state(cla_cpuhp_state);
+ cla_domains_free();
+ platform_driver_unregister(&cla_driver);
+}
+
+module_init(cla_module_init);
+module_exit(cla_module_exit);
MODULE_DESCRIPTION("Arm Core Local Accelerator");
MODULE_AUTHOR("Arm Limited");
diff --git a/drivers/misc/arm-cla/cla-topology.c b/drivers/misc/arm-cla/cla-topology.c
new file mode 100644
index 000000000000..5137a0e041e5
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-topology.c
@@ -0,0 +1,172 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm CLA driver - device topology initialization
+ *
+ * A CLA domain is a group of devices that work together and cannot be isolated
+ * from each other. They are owned by a single user at a time.
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/errno.h>
+#include <linux/gfp.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+
+#include "arm-cla.h"
+
+DEFINE_XARRAY(cla_domains);
+unsigned int cla_nr_domains;
+
+/*
+ * Some CPUs may not have a CLA. So cla_lut_cpu (indexed by CPU) may be sparse,
+ * but cla_lut_pg (indexed by page offset in mmap'ed file) is a contiguous array
+ * of all devices, of size cla_nr_devs, sorted by domain.
+ */
+struct cla_dev **cla_lut_cpu;
+struct cla_dev **cla_lut_pg;
+unsigned int cla_nr_devs;
+
+static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int fwid)
+{
+ int ret;
+ struct cla_domain *domain;
+
+ domain = kzalloc_obj(*domain);
+ if (!domain)
+ return ERR_PTR(-ENOMEM);
+
+ ret = xa_insert(&cla_domains, fwid, domain, GFP_KERNEL);
+ if (ret < 0)
+ goto err_free_domain;
+
+ domain->nr_devs = 1;
+ domain->devs = kzalloc_obj(*domain->devs);
+ if (!domain->devs) {
+ ret = -ENOMEM;
+ goto err_free_id;
+ }
+ domain->devs[0] = dev;
+
+ return domain;
+
+err_free_id:
+ xa_erase(&cla_domains, fwid);
+err_free_domain:
+ kfree(domain);
+ return ERR_PTR(ret);
+}
+
+/**
+ * cla_dev_domain_get - get or create a CLA domain for a device
+ * @dev: CLA device
+ *
+ * Return: CLA domain pointer on success, or an ERR_PTR() on failure.
+ */
+struct cla_domain *cla_dev_domain_get(struct cla_dev *dev)
+{
+ int ret;
+ unsigned int fwid;
+ struct cla_dev **devs;
+ struct cla_domain *domain;
+
+ /* Domain ID is provided by firmware */
+ ret = of_property_read_u32(dev->dev->of_node, "arm,domain", &fwid);
+ if (WARN_ON(ret))
+ return ERR_PTR(-EINVAL);
+
+ domain = xa_load(&cla_domains, fwid);
+ if (domain) {
+ devs = krealloc_array(domain->devs, domain->nr_devs + 1,
+ sizeof(*devs), GFP_KERNEL);
+ if (!devs)
+ return ERR_PTR(-ENOMEM);
+ domain->devs = devs;
+ domain->devs[domain->nr_devs++] = dev;
+ return domain;
+ }
+
+ return cla_domain_alloc(dev, fwid);
+}
+
+/**
+ * cla_domains_finalise - build CLA device lookup tables
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int __init cla_domains_finalise(void)
+{
+ int ret = -ENOMEM;
+ unsigned int j;
+ unsigned long fwid;
+ struct cla_domain *domain;
+ unsigned int pg_offset = 0;
+
+ cla_lut_cpu = kzalloc_objs(*cla_lut_cpu, nr_cpu_ids);
+ if (!cla_lut_cpu)
+ goto err_free;
+
+ cla_lut_pg = kzalloc_objs(*cla_lut_pg, cla_nr_devs);
+ if (!cla_lut_pg)
+ goto err_free;
+
+ ret = -EINVAL;
+ cla_nr_domains = 0;
+
+ /*
+ * Populate the lookup tables. Domains are keyed by fwid (which may be
+ * arbitrary) in xarray. Provide domain with a suitable, consecutive id
+ * for user space.
+ */
+ xa_for_each(&cla_domains, fwid, domain) {
+ domain->id = cla_nr_domains++;
+ domain->pg_offset = pg_offset;
+
+ for (j = 0; j < domain->nr_devs; j++) {
+ struct cla_dev *dev = domain->devs[j];
+
+ if (WARN_ON(dev->cpu >= nr_cpu_ids) ||
+ WARN_ON(pg_offset >= cla_nr_devs) ||
+ WARN_ON(cla_lut_cpu[dev->cpu]) ||
+ WARN_ON(cla_lut_pg[pg_offset]))
+ goto err_free;
+
+ cla_lut_cpu[dev->cpu] = dev;
+ cla_lut_pg[pg_offset] = dev;
+ dev->pg_offset = pg_offset;
+ pg_offset++;
+ }
+ }
+
+ return 0;
+
+err_free:
+ kfree(cla_lut_cpu);
+ kfree(cla_lut_pg);
+ cla_lut_cpu = NULL;
+ cla_lut_pg = NULL;
+ return ret;
+}
+
+/**
+ * cla_domains_free - free CLA domains and lookup tables
+ *
+ * Domain schedulers must be stopped and CPU hotplug callbacks removed first.
+ */
+void cla_domains_free(void)
+{
+ unsigned long fwid;
+ struct cla_domain *domain;
+
+ kfree(cla_lut_cpu);
+ kfree(cla_lut_pg);
+ cla_lut_cpu = NULL;
+ cla_lut_pg = NULL;
+ xa_for_each(&cla_domains, fwid, domain) {
+ xa_erase(&cla_domains, fwid);
+ kfree(domain->devs);
+ kfree(domain);
+ }
+ cla_nr_domains = 0;
+}
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 05/15] misc/arm-cla: Initialize devices on CPU bringup
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (3 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 04/15] misc/arm-cla: Probe firmware-described devices Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 06/15] misc/arm-cla: Accelerator context save and restore Ryan Roberts
` (9 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
CLA registers are only accessible from the CPU local to the CLA, so
perform device initialization from the CPU hotplug online callback.
Clear the DATA registers, reset each attached accelerator and verify
that it returns to the expected idle state. Record the set of usable
accelerators and expose only those through PL0CTRL, and through PL1CTRL
when the kernel is running at EL2.
Reset optional timestamp and PMU controls where available so userspace
starts from a predictable state, but do not fail probing if those
optional controls cannot be programmed.
On CPU teardown, reset all known accelerators and mark the device broken
if reset fails, since the hotplug teardown path cannot safely fail.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/arm-cla.h | 14 +++
drivers/misc/arm-cla/cla-init.c | 191 +++++++++++++++++++++++++++++++-
drivers/misc/arm-cla/cla-ops.c | 23 ++++
3 files changed, 227 insertions(+), 1 deletion(-)
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 61176c4fce0f..6890c813a5c3 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -27,16 +27,25 @@ struct cla_domain;
* @regs: Registers accessed by the kernel.
* @dev: The platform device.
* @pfn: Page of registers assigned to user.
+ * @accelerators: Available accelerators.
* @pg_offset: Mmap offset of this device.
* @domain: The domain this CLA belongs to.
+ *
+ * Mutable, protected by CPU hotplug locking:
+ * @discovered: Discovered state is cached.
+ * @broken: Hardware failure.
*/
struct cla_dev {
unsigned int cpu;
void __iomem *regs;
struct device *dev;
unsigned long pfn;
+ u8 accelerators;
unsigned long pg_offset;
struct cla_domain *domain;
+
+ bool discovered;
+ bool broken;
};
/**
@@ -61,6 +70,10 @@ extern struct cla_dev **cla_lut_cpu;
extern struct cla_dev **cla_lut_pg;
extern unsigned int cla_nr_devs;
+#define cla_for_each_accid(dev, accid) \
+ for ((accid) = 0; (accid) < CLA_NUM_ACC; (accid)++) \
+ for_each_if((dev)->accelerators & BIT(accid))
+
#define cla_dbg(_dev, fmt, ...) \
dev_dbg((_dev)->dev, "[%u] " fmt, (_dev)->cpu, ##__VA_ARGS__)
#define cla_info(_dev, fmt, ...) \
@@ -98,6 +111,7 @@ void cla_domains_free(void);
int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
int cla_op_reset(struct cla_dev *dev, unsigned int accid);
+int cla_op_reset_all(struct cla_dev *dev);
int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
size_t nregs, u64 *regs);
int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 6818fb8c38ed..2974bf71033c 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -18,18 +18,204 @@
static int cla_cpuhp_state = -1;
+static int cla_reset_ts(struct cla_dev *dev, unsigned int accid)
+{
+ int ret;
+ u64 reg;
+
+ ret = cla_op_regread(dev, accid, CLA_REG_ACAP, 1, ®);
+ if (ret)
+ return ret;
+ if (!FIELD_GET(CLA_ACAP_TS, reg))
+ return 0;
+
+ /*
+ * Disable TS control from userspace, provide TS from CNTP. If we do
+ * have to provide a timer to userspace or a virtual offset to a guest,
+ * we'll need to make sure we have access to both TSCTRLOWNER and
+ * TSOFFOWNER. For now best effort.
+ */
+ reg = FIELD_PREP(CLA_TSCTRLOWNER_PL, cla_kernel_pl);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSCTRLOWNER, 1, ®);
+ if (!ret) {
+ reg = FIELD_PREP(CLA_TSCTRL_TS, CLA_TSCTRL_PHYSICAL);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSCTRL, 1, ®);
+ if (ret)
+ return ret;
+ }
+
+ reg = FIELD_PREP(CLA_TSOFFOWNER_PL, cla_kernel_pl);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSOFFOWNER, 1, ®);
+ if (!ret) {
+ reg = 0;
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSVOFF, 1, ®);
+ if (ret)
+ return ret;
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSPOFF, 1, ®);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+static int cla_reset_pmu(struct cla_dev *dev, unsigned int accid)
+{
+ int ret;
+ u64 reg;
+
+ /* Disable PMU access */
+ reg = FIELD_PREP(CLA_PMUOWNER_PL, cla_kernel_pl);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_PMUOWNER, 1, ®);
+ if (!ret) {
+ reg = 0;
+ ret = cla_op_regwrite(dev, accid, CLA_REG_PMURESET, 1, ®);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+/*
+ * Discover immutable accelerator properties the first time the CPU comes
+ * online. Return: 0 on success, 1 if the accelerator is not usable, or an
+ * error.
+ */
+static int cla_dev_discover_accel(struct cla_dev *dev, unsigned int accid)
+{
+ dev->accelerators |= (1 << accid);
+
+ return 0;
+}
+
+/*
+ * Reset hardware on every online, but discover its properties only once.
+ * Return: 0 on success, 1 if the accelerator is not attached or not usable, or
+ * an error.
+ */
+static int cla_dev_setup_accel(struct cla_dev *dev, unsigned int accid)
+{
+ u64 status;
+ int ret;
+
+ /*
+ * Reset even accelerators excluded during discovery, so that they cannot
+ * issue memory transactions when we set up the shared MTC. Only skip
+ * absent accelerators that were not previously discovered. Fail if a
+ * higher PL has masked access.
+ */
+ ret = cla_op_reset(dev, accid);
+ if (ret > 0 && (dev->accelerators & (1 << accid))) {
+ cla_err(dev, "[%u] previously discovered accelerator is absent\n",
+ accid);
+ return -ENODEV;
+ }
+ if (ret)
+ return ret;
+
+ status = cla_reg_read(dev, CLA_REG_STATUS(accid));
+ if ((status & CLA_STATUS_STATE_MASK) != CLA_STATUS_STATE_IDLE) {
+ cla_err(dev, "unexpected status 0x%llx for accelerator %d\n",
+ status, accid);
+ return -EIO;
+ }
+
+ /*
+ * The following are nice to have, but the accelerator should work
+ * without them.
+ */
+ ret = cla_reset_ts(dev, accid);
+ if (ret)
+ cla_err(dev, "[%u] could not reset TS: %d\n", accid, ret);
+
+ ret = cla_reset_pmu(dev, accid);
+ if (ret)
+ cla_err(dev, "[%u] could not reset PMU: %d\n", accid, ret);
+
+ if (!dev->discovered) {
+ ret = cla_dev_discover_accel(dev, accid);
+ if (ret)
+ return ret;
+ } else if (!(dev->accelerators & (1 << accid))) {
+ return 1;
+ }
+
+ return 0;
+}
+
+/* Clean the device before releasing it */
+static void cla_dev_reinit(struct cla_dev *dev)
+{
+ int i;
+
+ if (dev->broken)
+ return;
+
+ if (WARN_ON(cla_op_reset_all(dev))) {
+ dev->broken = true;
+ return;
+ }
+
+ if (is_kernel_in_hyp_mode())
+ cla_reg_write(dev, CLA_REG_PL1CTRL, ~0ULL);
+ cla_reg_write(dev, CLA_REG_PL0CTRL, ~0ULL);
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), 0);
+ cla_reg_write(dev, CLA_REG_LRESP, 0);
+}
+
static int cla_dev_setup(unsigned int cpu)
{
+ int i;
+ int ret;
+ unsigned int accid;
struct cla_dev *dev;
+ u64 plxctrl_val = 0;
dev = cla_lut_cpu[cpu];
- if (!dev)
+ if (!dev || dev->broken)
return 0;
if (WARN_ON(smp_processor_id() != cpu || dev->cpu != cpu))
return -EINVAL;
+ if (!dev->discovered) {
+ /* Discard partial results from an earlier, failed discovery. */
+ dev->accelerators = 0;
+ }
+
+ /* Clear DATA and LRESP_DATANZ */
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), 0);
+
+ /*
+ * Reset all accelerators. We restrict PLxCTRL to the accelerators that
+ * are attached and well behaved.
+ */
+ for (accid = 0; accid < CLA_NUM_ACC; accid++) {
+ ret = cla_dev_setup_accel(dev, accid);
+ if (ret > 0)
+ continue;
+ else if (ret < 0)
+ goto err;
+
+ plxctrl_val |= CLA_PLxCTRL_PREP(accid,
+ FIELD_PREP(CLA_PLxCTRL_AVAIL, 1));
+ }
+
+ if (is_kernel_in_hyp_mode())
+ cla_reg_write(dev, CLA_REG_PL1CTRL, plxctrl_val);
+
+ cla_reg_write(dev, CLA_REG_PL0CTRL, plxctrl_val);
+
+ if (dev->accelerators)
+ cla_info(dev, "available accelerators: 0x%02x\n",
+ dev->accelerators);
+
+ dev->discovered = true;
return 0;
+err:
+ cla_dev_reinit(dev);
+ return ret;
}
static int cla_dev_teardown(unsigned int cpu)
@@ -44,6 +230,8 @@ static int cla_dev_teardown(unsigned int cpu)
if (!dev)
return 0;
+ cla_dev_reinit(dev);
+
return 0;
}
@@ -85,6 +273,7 @@ static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
dev->cpu = cpu;
dev->dev = parent;
+
/* Attempt to find device domain, or allocate a new one */
dev->domain = cla_dev_domain_get(dev);
if (IS_ERR(dev->domain))
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
index 46026304a057..5d053c6aa1f7 100644
--- a/drivers/misc/arm-cla/cla-ops.c
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -207,6 +207,29 @@ int cla_op_reset(struct cla_dev *dev, unsigned int accid)
return 0;
}
+/**
+ * cla_op_reset_all - Reset all attached accelerators
+ * @dev: CLA device.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_op_reset_all(struct cla_dev *dev)
+{
+ int ret, err = 0;
+ unsigned int accid;
+
+ cla_for_each_accid(dev, accid) {
+ ret = cla_op_reset(dev, accid);
+ if (ret && !err)
+ err = ret;
+ }
+ /*
+ * Since all accelerators in this loop are present, cla_op_reset()
+ * returning 1 is an error.
+ */
+ return err <= 0 ? err : -ENODEV;
+}
+
/* regidx is unsigned long here to avoid overflow during calculations */
static int cla_op_access_reg(struct cla_dev *dev, u8 op,
enum cla_launch_data_mode data_mode,
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 06/15] misc/arm-cla: Accelerator context save and restore
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (4 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 05/15] misc/arm-cla: Initialize devices on CPU bringup Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 07/15] misc/arm-cla: Set up memory translation context Ryan Roberts
` (8 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Add helpers to save and restore CLA context for later domain switching.
The saved state includes DATA registers, LRESP, per-accelerator SRSTATE
and any implementation-defined internal accelerator state exposed
through the IAS register window.
Use ENTERSR to stop each accelerator and collect SRSTATE, then read the
advertised internal register state with REGREAD. Restore by entering SR
mode from a reset, idle accelerator, writing the saved internal state
with REGWRITE and completing the transition with EXITSR.
Reject accelerators that require SAVE/RESTORE operations during setup,
since that path needs a DMA buffer for state storage. Allocate
per-device save areas for all CLAs in a domain, sized from the combined
IASSIZE discovered at setup time.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Makefile | 1 +
drivers/misc/arm-cla/arm-cla.h | 32 ++++
drivers/misc/arm-cla/cla-init.c | 35 ++++
drivers/misc/arm-cla/cla-ops.c | 49 ++++++
drivers/misc/arm-cla/cla-regs.c | 295 ++++++++++++++++++++++++++++++++
5 files changed, 412 insertions(+)
create mode 100644 drivers/misc/arm-cla/cla-regs.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index 84ed1a054a21..c1b14155454d 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -3,6 +3,7 @@
arm-cla-y := \
cla-init.o \
cla-ops.o \
+ cla-regs.o \
cla-topology.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 6890c813a5c3..6d72d57e5422 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -29,6 +29,7 @@ struct cla_domain;
* @pfn: Page of registers assigned to user.
* @accelerators: Available accelerators.
* @pg_offset: Mmap offset of this device.
+ * @iassizes: Combined regstate of all accels in bytes.
* @domain: The domain this CLA belongs to.
*
* Mutable, protected by CPU hotplug locking:
@@ -42,6 +43,7 @@ struct cla_dev {
unsigned long pfn;
u8 accelerators;
unsigned long pg_offset;
+ unsigned long iassizes;
struct cla_domain *domain;
bool discovered;
@@ -64,6 +66,27 @@ struct cla_domain {
struct cla_dev **devs;
};
+/**
+ * struct cla_regs_pl0 - Saved CLA register state (PL0)
+ *
+ * @data: DATA registers.
+ * @lresp: LRESP register.
+ * @accel_valid: Accelerator state has been saved once.
+ * @srstate: Save/restore state for each accelerator.
+ * @regstate: Internal accelerator state.
+ */
+struct cla_regs_pl0 {
+ u64 data[CLA_NUM_DATA_REGS];
+ u64 lresp;
+ bool accel_valid;
+ u64 srstate[CLA_NUM_ACC][CLA_SRSTATE_LEN];
+ u64 regstate[];
+};
+
+struct cla_ctx {
+ void **regs;
+};
+
extern struct xarray cla_domains;
extern unsigned int cla_nr_domains;
extern struct cla_dev **cla_lut_cpu;
@@ -116,5 +139,14 @@ int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
size_t nregs, u64 *regs);
int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
unsigned int regidx, size_t nregs, u64 *regs);
+int cla_op_entersr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
+int cla_op_exitsr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
+
+int cla_regs_install(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id);
+int cla_regs_uninstall(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id);
+int cla_ctx_alloc_regs(struct cla_domain *domain, struct cla_ctx *ctx);
+void cla_ctx_free_regs(struct cla_domain *domain, struct cla_ctx *ctx);
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 2974bf71033c..a4d2ffaa3136 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -7,8 +7,10 @@
#include <linux/cpuhotplug.h>
#include <linux/cpuhplock.h>
+#include <linux/math.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/overflow.h>
#include <linux/platform_device.h>
#include <linux/smp.h>
@@ -82,6 +84,38 @@ static int cla_reset_pmu(struct cla_dev *dev, unsigned int accid)
*/
static int cla_dev_discover_accel(struct cla_dev *dev, unsigned int accid)
{
+ unsigned long iassizes;
+ u64 iassize;
+ u64 acap;
+ int ret;
+
+ /*
+ * This driver supports context switching with REGREAD and REGWRITE.
+ * SROP (SAVE and RESTORE ops) requires DMA-backed state storage, which
+ * is not implemented here.
+ */
+ ret = cla_op_regread(dev, accid, CLA_REG_ACAP, 1, &acap);
+ if (ret)
+ return ret;
+ if (FIELD_GET(CLA_ACAP_SROP, acap)) {
+ cla_err(dev, "[%u] SROP not supported\n", accid);
+ return 1;
+ }
+
+ ret = cla_op_regread(dev, accid, CLA_REG_IASSIZE, 1, &iassize);
+ if (ret)
+ return ret;
+ if (FIELD_GET(CLA_ACAP_REGSTATE, acap)) {
+ /* Reserve whole u64 entries for each accelerator's state. */
+ if (check_add_overflow(iassize, sizeof(u64) - 1, &iassize) ||
+ check_add_overflow(dev->iassizes, round_down(iassize,
+ sizeof(u64)), &iassizes)) {
+ cla_err(dev, "[%u] IASSIZE total overflows\n", accid);
+ return -EOVERFLOW;
+ }
+ dev->iassizes = iassizes;
+ }
+
dev->accelerators |= (1 << accid);
return 0;
@@ -181,6 +215,7 @@ static int cla_dev_setup(unsigned int cpu)
if (!dev->discovered) {
/* Discard partial results from an earlier, failed discovery. */
dev->accelerators = 0;
+ dev->iassizes = 0;
}
/* Clear DATA and LRESP_DATANZ */
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
index 5d053c6aa1f7..bb0bac5c2d35 100644
--- a/drivers/misc/arm-cla/cla-ops.c
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -314,3 +314,52 @@ int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
return cla_op_access_reg(dev, CLA_LAUNCH_OP_REGWRITE, CLA_DATA_IN,
accid, regidx, nregs, regs);
}
+
+/**
+ * cla_op_entersr - launch ENTERSR operation.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @srstate: 64-byte save/restore state written by CLA. May be NULL if the
+ * SRSTATE isn't needed.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_entersr(struct cla_dev *dev, unsigned int accid, u64 *srstate)
+{
+ int ret;
+ u64 status;
+ struct cla_launch launch = {
+ .op = CLA_LAUNCH_OP_ENTERSR,
+ .ndata_m1 = 7,
+ .accid = accid,
+ .data_mode = srstate ? CLA_DATA_OUT : CLA_DATA_NONE,
+ .data = srstate,
+ };
+
+ ret = cla_op_launch(dev, &launch);
+ if (ret)
+ return ret;
+
+ return cla_wait_status_dmb(dev, accid, &status);
+}
+
+/**
+ * cla_op_exitsr - launch EXITSR operation.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @srstate: 64-byte save/restore state written to CLA.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_exitsr(struct cla_dev *dev, unsigned int accid, u64 *srstate)
+{
+ struct cla_launch launch = {
+ .op = CLA_LAUNCH_OP_EXITSR,
+ .ndata_m1 = 7,
+ .accid = accid,
+ .data_mode = CLA_DATA_IN,
+ .data = srstate,
+ };
+
+ return cla_op_launch(dev, &launch);
+}
diff --git a/drivers/misc/arm-cla/cla-regs.c b/drivers/misc/arm-cla/cla-regs.c
new file mode 100644
index 000000000000..2452a7601990
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-regs.c
@@ -0,0 +1,295 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - save/restore accelerator context
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/io.h>
+#include <linux/overflow.h>
+
+#include "arm-cla.h"
+
+static int cla_regs_save_accel(struct cla_dev *dev, unsigned int accid,
+ struct cla_regs_pl0 *regs, size_t *regstate_off)
+{
+ int ret;
+ u64 status;
+ size_t regstate_size;
+ u64 *srstate = regs->srstate[accid];
+ size_t nr_regstate = dev->iassizes / sizeof(u64);
+
+ ret = cla_op_entersr(dev, accid, srstate);
+ if (ret) {
+ /*
+ * Note that we don't expect the accelerator to return an error.
+ * Implementations that don't support context switch cancel the
+ * work and write an error in SRSTATE_STATUS, but don't fail
+ * ENTERSR.
+ */
+ return ret;
+ }
+
+ status = cla_reg_read(dev, CLA_REG_STATUS(accid));
+ if ((status & CLA_STATUS_STATE_MASK) != CLA_STATUS_STATE_SRMODE) {
+ cla_err(dev, "unexpected SR status 0x%llx\n", status);
+ return -EIO;
+ }
+
+ /*
+ * A device that supports SROP fails device probe. Others should not
+ * report SROP here, we don't support it.
+ */
+ WARN_ON(FIELD_GET(CLA_SRSTATE_0_SROP, srstate[0]));
+
+ regstate_size = FIELD_GET(CLA_SRSTATE_0_REGSTATE, srstate[0]);
+ if (WARN_ON(*regstate_off > nr_regstate ||
+ regstate_size > nr_regstate - *regstate_off))
+ return -ENOSPC;
+
+ if (regstate_size) {
+ ret = cla_op_regread(dev, accid, CLA_REG_IASn, regstate_size,
+ regs->regstate + *regstate_off);
+ if (ret) {
+ cla_err(dev, "failed to save regstate: %d\n", ret);
+ return ret;
+ }
+ }
+
+ *regstate_off += regstate_size;
+
+ return 0;
+}
+
+/*
+ * Errors are very unlikely, but if they happen the device is left in SRMODE.
+ */
+static int cla_regs_restore_accel(struct cla_dev *dev, unsigned int accid,
+ struct cla_regs_pl0 *regs,
+ size_t *regstate_off)
+{
+ int ret;
+ u64 status;
+ size_t regstate_size;
+ u64 *srstate = regs->srstate[accid];
+ size_t nr_regstate = dev->iassizes / sizeof(u64);
+
+ /*
+ * The accelerator was reset and isn't in SRMODE. Later we could support
+ * coming directly from cla_regs_save_accel() in SRMODE, but at the
+ * moment we always need a RESET.
+ */
+ status = cla_reg_read(dev, CLA_REG_STATUS(accid));
+ if ((status & CLA_STATUS_STATE_MASK) != CLA_STATUS_STATE_IDLE) {
+ cla_err(dev, "unexpected status 0x%llx\n", status);
+ return -EIO;
+ }
+
+ ret = cla_op_entersr(dev, accid, NULL);
+ if (ret)
+ return ret;
+
+ WARN_ON(FIELD_GET(CLA_SRSTATE_0_SROP, srstate[0]));
+
+ regstate_size = FIELD_GET(CLA_SRSTATE_0_REGSTATE, srstate[0]);
+ if (WARN_ON(*regstate_off > nr_regstate ||
+ regstate_size > nr_regstate - *regstate_off))
+ return -ENOSPC;
+
+ if (regstate_size) {
+ ret = cla_op_regwrite(dev, accid, CLA_REG_IASn, regstate_size,
+ regs->regstate + *regstate_off);
+ if (ret) {
+ cla_err(dev, "failed to restore regstate: %d\n", ret);
+ return ret;
+ }
+ }
+
+ *regstate_off += regstate_size;
+
+ return cla_op_exitsr(dev, accid, srstate);
+}
+
+static int cla_regs_uninstall_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id)
+{
+ int i;
+ int ret;
+ unsigned int accid;
+ size_t regstate_off = 0;
+ struct cla_regs_pl0 *regs = ctx->regs[ctx_id];
+
+ /*
+ * When we interrupt the user process in the middle of launching a
+ * command, we have to wait for LRESP_PENDING to clear before we can
+ * launch a new command or save the DATA registers.
+ */
+ ret = cla_op_wait_lresp(dev, ®s->lresp);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ regs->data[i] = cla_reg_read(dev, CLA_REG_DATA(i));
+
+ cla_for_each_accid(dev, accid) {
+ ret = cla_regs_save_accel(dev, accid, regs, ®state_off);
+ if (ret)
+ return ret;
+ }
+
+ regs->accel_valid = true;
+
+ /*
+ * "If the accelerator is non idle when ENTERSR is launched, then a
+ * RESET operation is required after the internal accelerator state has
+ * been saved"
+ *
+ * Reset unconditionally to establish a clean state before assigning
+ * the next context.
+ */
+ return cla_op_reset_all(dev);
+}
+
+static int cla_regs_install_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id)
+{
+ int i;
+ int ret;
+ unsigned int accid;
+ size_t regstate_off = 0;
+ struct cla_regs_pl0 *regs = ctx->regs[ctx_id];
+
+ cla_for_each_accid(dev, accid) {
+ if (regs->accel_valid) {
+ ret = cla_regs_restore_accel(dev, accid, regs, ®state_off);
+ if (ret)
+ return ret;
+ /*
+ * At this point we must not read STATUS because that
+ * would clear EVENT. Userspace always gets a spurious
+ * EVENT on restore, because we have no way to
+ * save/restore EVENT and userspace missing an event
+ * would be worse than getting a spurious one.
+ */
+ } else {
+ /*
+ * If this context has never been scheduled, then we
+ * just clean the CLA regs. Also clear EVENT by reading
+ * STATUS, to provide a pristine context.
+ */
+ cla_reg_read(dev, CLA_REG_STATUS(accid));
+ }
+ }
+
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), regs->data[i]);
+ cla_reg_write(dev, CLA_REG_LRESP, regs->lresp);
+
+ return 0;
+}
+
+/**
+ * cla_regs_install - Install and/or restore CLA context
+ * @dev: CLA device
+ * @ctx: CLA context to install
+ * @ctx_id: CLA context number within the domain
+ *
+ * Save the kernel context if necessary, and install a user context.
+ *
+ * PL0 context: Restore the DATA and LRESP registers, and accelerator state if
+ * one has been saved.
+ *
+ * This function is called with all accelerators idle and no trace of previous
+ * work.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_regs_install(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id)
+{
+ return cla_regs_install_pl0(dev, ctx, ctx_id);
+}
+
+/**
+ * cla_regs_uninstall - Remove and/or save CLA context
+ * @dev: CLA device
+ * @ctx: CLA context to remove
+ * @ctx_id: CLA context number within the domain
+ *
+ * Uninstall a user context, and restore the kernel context if necessary.
+ *
+ * PL0 context: Save the DATA and LRESP registers and accelerator state.
+ *
+ * When this completes, all accelerators are idle. If this fails, some
+ * accelerators may still be running.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_regs_uninstall(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id)
+{
+ return cla_regs_uninstall_pl0(dev, ctx, ctx_id);
+}
+
+/**
+ * cla_ctx_alloc_regs - Allocate register state for a CLA context
+ * @domain: CLA domain
+ * @ctx: CLA context
+ *
+ * Allocate register state to save and restore every device in @domain, store it
+ * in @ctx.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_ctx_alloc_regs(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ void **regs_ptrs;
+ size_t size;
+ int i;
+
+ regs_ptrs = kmalloc_objs(*regs_ptrs, domain->nr_devs,
+ GFP_KERNEL_ACCOUNT);
+ if (!regs_ptrs)
+ return -ENOMEM;
+
+ for (i = 0; i < domain->nr_devs; i++) {
+ struct cla_dev *dev = domain->devs[i];
+ size_t nr_regstate = dev->iassizes / sizeof(u64);
+ void *regs;
+
+ /*
+ * The regs structures are only ever accessed from the CLA
+ * device's CPU, so try to allocate them on the right NUMA node
+ */
+ size = struct_size_t(struct cla_regs_pl0, regstate, nr_regstate);
+ regs = kvzalloc_node(size, GFP_KERNEL_ACCOUNT,
+ cpu_to_node(dev->cpu));
+ if (!regs)
+ goto err_free;
+
+ regs_ptrs[i] = regs;
+ }
+
+ ctx->regs = regs_ptrs;
+ return 0;
+
+err_free:
+ for (i--; i >= 0; i--)
+ kvfree(regs_ptrs[i]);
+ kfree(regs_ptrs);
+ return -ENOMEM;
+}
+
+/**
+ * cla_ctx_free_regs - Free register state for a CLA context
+ * @domain: CLA domain
+ * @ctx: CLA context
+ */
+void cla_ctx_free_regs(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ int i;
+
+ for (i = 0; i < domain->nr_devs; i++)
+ kvfree(ctx->regs[i]);
+ kfree(ctx->regs);
+}
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 07/15] misc/arm-cla: Set up memory translation context
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (5 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 06/15] misc/arm-cla: Accelerator context save and restore Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 08/15] misc/arm-cla: Manage domain contexts Ryan Roberts
` (7 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
CLA accelerators use a Memory Translation Context to describe the
page-table and memory-system configuration used for DMA through the CPU
MMU. Set up the kernel-owned MTC during CLA bringup so later context
switches only need to replace the user TTBR0 value.
Populate the MTC from the CPU EL1 configuration, including SCTLR, MAIR,
TCR and TCR2 where present. Configure the accelerator to run as EL0, put
the ASID in TTBR0 and disable TTBR1 translations because the driver only
installs user contexts in the TTBR0 range. When running at EL2, also
program the host HCR state.
Add SETCTX and GETCTX launch helpers for MTC register access. Use the
first available accelerator ID for these operations, since the MTC is
shared by the CLA but the operation still requires a valid accelerator
ID.
Clear the MTC on CPU teardown, and add helpers to install and uninstall
a user translation context by updating TTBR0 with the process page
table, ASID and CnP state.
Co-developed-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
drivers/misc/arm-cla/Makefile | 1 +
drivers/misc/arm-cla/arm-cla.h | 24 ++++
drivers/misc/arm-cla/cla-init.c | 10 +-
drivers/misc/arm-cla/cla-mtc.c | 206 ++++++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-ops.c | 52 ++++++++
5 files changed, 291 insertions(+), 2 deletions(-)
create mode 100644 drivers/misc/arm-cla/cla-mtc.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index c1b14155454d..df3ad7e4a1b4 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -2,6 +2,7 @@
arm-cla-y := \
cla-init.o \
+ cla-mtc.o \
cla-ops.o \
cla-regs.o \
cla-topology.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 6d72d57e5422..40e95d3128ad 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -83,8 +83,20 @@ struct cla_regs_pl0 {
u64 regstate[];
};
+/**
+ * struct cla_ctx_key - Unique context identifier
+ * @mm: For PL0 contexts, the mm.
+ * @file: for PL0 contexts, the file.
+ */
+struct cla_ctx_key {
+ struct mm_struct *mm;
+ struct file *file;
+};
+
struct cla_ctx {
+ struct cla_ctx_key key;
void **regs;
+ unsigned int asid;
};
extern struct xarray cla_domains;
@@ -139,6 +151,10 @@ int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
size_t nregs, u64 *regs);
int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
unsigned int regidx, size_t nregs, u64 *regs);
+int cla_op_setctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs);
+int cla_op_getctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs);
int cla_op_entersr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
int cla_op_exitsr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
@@ -149,4 +165,12 @@ int cla_regs_uninstall(struct cla_dev *dev, struct cla_ctx *ctx,
int cla_ctx_alloc_regs(struct cla_domain *domain, struct cla_ctx *ctx);
void cla_ctx_free_regs(struct cla_domain *domain, struct cla_ctx *ctx);
+int cla_mtc_setup(struct cla_dev *dev);
+int cla_mtc_clear(struct cla_dev *dev);
+int cla_mtc_install(struct cla_dev *dev, struct cla_ctx *ctx);
+int cla_mtc_uninstall(struct cla_dev *dev, struct cla_ctx *ctx);
+int cla_mtc_get_ctx(struct cla_ctx *ctx);
+void cla_mtc_put_ctx(struct cla_ctx *ctx);
+void cla_mtc_leak_ctx(struct cla_ctx *ctx);
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index a4d2ffaa3136..58f37787a59c 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -184,7 +184,8 @@ static void cla_dev_reinit(struct cla_dev *dev)
if (dev->broken)
return;
- if (WARN_ON(cla_op_reset_all(dev))) {
+ if (WARN_ON(cla_op_reset_all(dev)) ||
+ WARN_ON(cla_mtc_clear(dev))) {
dev->broken = true;
return;
}
@@ -242,9 +243,14 @@ static int cla_dev_setup(unsigned int cpu)
cla_reg_write(dev, CLA_REG_PL0CTRL, plxctrl_val);
- if (dev->accelerators)
+ if (dev->accelerators) {
+ ret = cla_mtc_setup(dev);
+ if (ret)
+ goto err;
+
cla_info(dev, "available accelerators: 0x%02x\n",
dev->accelerators);
+ }
dev->discovered = true;
return 0;
diff --git a/drivers/misc/arm-cla/cla-mtc.c b/drivers/misc/arm-cla/cla-mtc.c
new file mode 100644
index 000000000000..9ab25085dfdf
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-mtc.c
@@ -0,0 +1,206 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - memory translation context
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <asm/kvm_arm.h>
+#include <linux/mmu_context.h>
+
+#include "arm-cla.h"
+
+#define cla_kernel_mtc \
+ (cla_kernel_pl == 2 ? CLA_MTC_REGIDX_PL2 : CLA_MTC_REGIDX_PL1)
+
+/**
+ * cla_mtc_setup - initialize the memory translation context
+ * @dev: CLA device
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_setup(struct cla_dev *dev)
+{
+ int ret;
+ u64 tcr;
+ u64 regs[2 * CLA_MTC_PL_SIZE] = {};
+ u64 *kregs = ®s[cla_kernel_mtc];
+ size_t regs_size = cla_kernel_pl * CLA_MTC_PL_SIZE;
+ u64 reserved_ttbr = phys_to_ttbr(page_to_phys(ZERO_PAGE(0)));
+
+ /*
+ * The MTC registers deliberately have the same layout as their arm64
+ * sysreg counterparts. Where MTC registers don't define a particular
+ * field, it is specified as "reserved RAZ/WI", so we can safely copy
+ * the whole arm64 register value to the MTC register without needing to
+ * do any masking. That just leaves the possibility of the CPU
+ * supporting a VMSA feature which is necessary for correct sharing of
+ * pgtables that the MTC doesn't support (e.g. MTE, PIE, POE);
+ * Fortunately, CLA is part of the CPU implementation so this is
+ * guaranteed never to happen.
+ */
+
+ /* The accelerator always runs as EL0 */
+ kregs[CLA_MTC_PSTATE] = FIELD_PREP(CLA_MTC_PSTATE_EL, 0) |
+ FIELD_PREP(CLA_MTC_PSTATE_PAN,
+ system_uses_hw_pan());
+ kregs[CLA_MTC_TTBR0] = reserved_ttbr;
+ kregs[CLA_MTC_TTBR1] = reserved_ttbr;
+ kregs[CLA_MTC_SCTLR] = read_sysreg(sctlr_el1);
+ kregs[CLA_MTC_MAIR] = read_sysreg(mair_el1);
+
+ if (cpus_have_final_cap(ARM64_HAS_TCR2))
+ kregs[CLA_MTC_TCR2] = read_sysreg_s(SYS_TCR2_EL1);
+
+ tcr = read_sysreg(tcr_el1);
+ /*
+ * Put ASID in TTBR0, so that we only have one reg to update when
+ * switching context.
+ */
+ FIELD_MODIFY(TCR_EL1_A1, &tcr, 0);
+ /*
+ * We never map anything in the TTBR1 VA range so explicitly disable
+ * translations via TTBR1.
+ */
+ FIELD_MODIFY(TCR_EL1_EPD1_MASK, &tcr, 1);
+ kregs[CLA_MTC_TCR] = tcr;
+
+ if (cla_kernel_pl == 2) {
+ /* E2H and TGE */
+ regs[CLA_MTC_HCR_EL2] = HCR_HOST_VHE_FLAGS;
+ }
+
+ /*
+ * All accelerators are idle, meaning there can't be any memory
+ * transactions happening. So it is safe to setup the MTC in any order.
+ * i.e. there is no issue with setting SCTLR.M=1 before we have
+ * configured MAIR or TCR2.
+ */
+ ret = cla_op_setctx(dev, CLA_MTC_REGIDX_PL1, regs_size, regs);
+ if (ret) {
+ WARN_ON(cla_mtc_clear(dev));
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * cla_mtc_clear - clear the memory translation context
+ * @dev: CLA device
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_clear(struct cla_dev *dev)
+{
+ /* Don't set the RES1 bit in VTCR_EL2 because in MTC they are RAZ/WI. */
+ u64 regs[2 * CLA_MTC_PL_SIZE] = {};
+ size_t regs_size = cla_kernel_pl * CLA_MTC_PL_SIZE;
+
+ /* SETCTX fails if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ /*
+ * All accelerators are idle, meaning there can't be any memory
+ * transactions happening. So it is safe to clear the MTC in any order.
+ * i.e. there is no issue with clearing TTBRx before setting SCTLR.M=0.
+ */
+ return cla_op_setctx(dev, CLA_MTC_REGIDX_PL1, regs_size, regs);
+}
+
+static int cla_mtc_install_s1(struct cla_dev *dev, struct cla_ctx *ctx)
+{
+ u64 ttbr0;
+
+ /* SETCTX fails if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ ttbr0 = phys_to_ttbr(virt_to_phys(ctx->key.mm->pgd)) |
+ FIELD_PREP(TTBRx_EL1_ASID_MASK, ctx->asid);
+
+ if (system_supports_cnp())
+ ttbr0 |= TTBRx_EL1_CnP;
+
+ return cla_op_setctx(dev, cla_kernel_mtc + CLA_MTC_TTBR0, 1, &ttbr0);
+}
+
+static int cla_mtc_uninstall_s1(struct cla_dev *dev)
+{
+ u64 ttbr0 = phys_to_ttbr(page_to_phys(ZERO_PAGE(0)));
+
+ /* SETCTX fails if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ return cla_op_setctx(dev, cla_kernel_mtc + CLA_MTC_TTBR0, 1, &ttbr0);
+}
+
+/**
+ * cla_mtc_install - enable a translation context
+ * @dev: CLA device
+ * @ctx: context to install
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_install(struct cla_dev *dev, struct cla_ctx *ctx)
+{
+ return cla_mtc_install_s1(dev, ctx);
+}
+
+/**
+ * cla_mtc_uninstall - disable the translation context
+ * @dev: CLA device
+ * @ctx: context to remove
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_uninstall(struct cla_dev *dev, struct cla_ctx *ctx)
+{
+ return cla_mtc_uninstall_s1(dev);
+}
+
+/**
+ * cla_mtc_get_ctx - Get translation information
+ * @ctx: context to get
+ *
+ * Grab the information needed for installing this context.
+ * For stage 1 MTC, pin the ASID.
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_get_ctx(struct cla_ctx *ctx)
+{
+ ctx->asid = arm64_mm_context_get(ctx->key.mm);
+ if (ctx->asid == 0)
+ return -ENOSPC;
+
+ return 0;
+}
+
+/**
+ * cla_mtc_put_ctx - Release translation information
+ * @ctx: context to release
+ *
+ * Undo cla_mtc_get_ctx().
+ */
+void cla_mtc_put_ctx(struct cla_ctx *ctx)
+{
+ arm64_mm_context_put(ctx->key.mm);
+}
+
+/**
+ * cla_mtc_leak_ctx - Leak the memory translation config
+ * @ctx: the stuck context
+ *
+ * On catastrophic hardware failure, when we lose control of the device and
+ * cannot uninstall the context, take a reference to the memory context so that
+ * it doesn't get freed. The page directory may be accessed by the hardware
+ * sometime in the future and must not be reallocated.
+ */
+void cla_mtc_leak_ctx(struct cla_ctx *ctx)
+{
+ WARN(1, "leaking PL0 context\n");
+ mmgrab(ctx->key.mm);
+}
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
index bb0bac5c2d35..ceca84fd58a9 100644
--- a/drivers/misc/arm-cla/cla-ops.c
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -249,6 +249,10 @@ static int cla_op_access_reg(struct cla_dev *dev, u8 op,
case CLA_LAUNCH_OP_REGWRITE:
max_regidx = 0x100000000;
break;
+ case CLA_LAUNCH_OP_SETCTX:
+ case CLA_LAUNCH_OP_GETCTX:
+ max_regidx = 0x80;
+ break;
default:
WARN_ON(1);
return -EINVAL;
@@ -315,6 +319,54 @@ int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
accid, regidx, nregs, regs);
}
+/**
+ * cla_op_setctx - launch SETCTX operation.
+ * @dev: CLA device.
+ * @regidx: first MTC register index.
+ * @nregs: number of registers to set. Can be greater than 8 (accessed with
+ * multiple SETCTX operations).
+ * @regs: register content, array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_setctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs)
+{
+ /*
+ * Even though the MTC is shared between all accelerators, accid must be
+ * valid. If no accelerator is attached, the op fails.
+ */
+ if (!dev->accelerators)
+ return -ENODEV;
+
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_SETCTX, CLA_DATA_IN,
+ __ffs(dev->accelerators), regidx, nregs, regs);
+}
+
+/**
+ * cla_op_getctx - launch GETCTX operation.
+ * @dev: CLA device.
+ * @regidx: first MTC register index.
+ * @nregs: number of registers to get. Can be greater than 8 (accessed with
+ * multiple GETCTX operations).
+ * @regs: register content, array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_getctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs)
+{
+ /*
+ * Even though the MTC is shared between all accelerators, accid must be
+ * valid. If no accelerator is attached, the op fails.
+ */
+ if (!dev->accelerators)
+ return -ENODEV;
+
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_GETCTX, CLA_DATA_OUT,
+ __ffs(dev->accelerators), regidx, nregs, regs);
+}
+
/**
* cla_op_entersr - launch ENTERSR operation.
* @dev: CLA device.
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 08/15] misc/arm-cla: Manage domain contexts
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (6 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 07/15] misc/arm-cla: Set up memory translation context Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 09/15] misc/arm-cla: Add userspace interface Ryan Roberts
` (6 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
A CLA domain cannot be used by multiple address spaces concurrently
because its devices share a memory translation context and may
communicate without isolation. Represent each user of a domain with a
context keyed by the mm_struct and device file description, and track
contexts in a per-domain hash table.
Add a domain scheduler that queues contexts for assignment and gives
each one a fixed time slice. Before switching, revoke the outgoing
context's mappings from all VMAs for the device file. Then save its
accelerator state and remove its MTC, before installing the incoming
context's MTC and restoring the incoming accelerator state.
Run hardware operations through CPU-bound kthread workers, since each
CLA can only be accessed from its local CPU. Coordinate the per-device
workers with a domain worker so that every CLA in the domain is switched
as a single assignment unit.
Treat a failure on any device as a failure of the whole domain. Stop
further assignments and wake all waiters when a domain becomes unusable.
Co-developed-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
drivers/misc/arm-cla/Makefile | 2 +
drivers/misc/arm-cla/arm-cla.h | 118 +++++++
drivers/misc/arm-cla/cla-ctx.c | 146 +++++++++
drivers/misc/arm-cla/cla-init.c | 62 ++++
drivers/misc/arm-cla/cla-regs.c | 26 +-
drivers/misc/arm-cla/cla-sched.c | 473 ++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 23 ++
7 files changed, 841 insertions(+), 9 deletions(-)
create mode 100644 drivers/misc/arm-cla/cla-ctx.c
create mode 100644 drivers/misc/arm-cla/cla-sched.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index df3ad7e4a1b4..a92c3594e5c9 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -2,9 +2,11 @@
arm-cla-y := \
cla-init.o \
+ cla-ctx.o \
cla-mtc.o \
cla-ops.o \
cla-regs.o \
+ cla-sched.o \
cla-topology.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 40e95d3128ad..a797330956d9 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -7,8 +7,19 @@
#ifndef _ARM_CLA_H_
#define _ARM_CLA_H_
+#include <linux/atomic.h>
+#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/io.h>
+#include <linux/kref.h>
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/mm.h>
+#include <linux/mm_types.h>
+#include <linux/mutex.h>
+#include <linux/rhashtable-types.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
#include "arm-cla-regs.h"
@@ -17,8 +28,20 @@
#define CLA_NUM_DATA_REGS 8
#define CLA_SRSTATE_LEN 8
+/* Quantum of CLA assignment */
+#define CLA_SLICE_MS 100
+
struct cla_domain;
+struct cla_call_on_cpu {
+ int ret;
+ struct {
+ struct cla_ctx *prev_ctx;
+ struct cla_ctx *next_ctx;
+ } sched;
+ struct kthread_work switch_ctx;
+};
+
/**
* struct cla_dev - CLA device
*
@@ -35,6 +58,8 @@ struct cla_domain;
* Mutable, protected by CPU hotplug locking:
* @discovered: Discovered state is cached.
* @broken: Hardware failure.
+ * @worker: CPU-bound worker to communicate with CLA.
+ * @call: Scheduling work.
*/
struct cla_dev {
unsigned int cpu;
@@ -48,22 +73,45 @@ struct cla_dev {
bool discovered;
bool broken;
+ struct kthread_worker *worker;
+ struct cla_call_on_cpu call;
};
/**
* struct cla_domain - Collection of cla_dev
*
+ * The whole domain is assigned to a single cla_ctx at a time.
+ *
* Immutable state:
* @id: Consecutive userspace domain identifier.
* @pg_offset: Mmap offset of the first device.
* @nr_devs: Number of devices in the domain.
* @devs: Devices.
+ *
+ * Mutable, only accessed under @lock:
+ * @lock: Protects the following members.
+ * @ctxs: All live contexts, keyed on cla_ctx_key.
+ * @queued_ctxs: Queue of contexts waiting for assignment.
+ * @worker: Kthread worker to coordinate reassignment.
+ * @reassign: Delayed work that switches contexts with time slicing.
+ *
+ * Mutable, some reads outside the lock:
+ * @broken: Hardware failure in any device in the domain.
+ * @assigned_ctx: Context to which domain is currently assigned.
*/
struct cla_domain {
unsigned int id;
unsigned long pg_offset;
unsigned int nr_devs;
struct cla_dev **devs;
+
+ struct mutex lock;
+ bool broken;
+ struct rhashtable ctxs;
+ struct list_head queued_ctxs;
+ struct cla_ctx *assigned_ctx;
+ struct kthread_worker *worker;
+ struct kthread_delayed_work reassign;
};
/**
@@ -93,10 +141,48 @@ struct cla_ctx_key {
struct file *file;
};
+#define CLA_CTX_KEY_PL0(_mm, _file) \
+ { \
+ .mm = (_mm), \
+ .file = (_file), \
+ }
+
+/**
+ * struct cla_ctx - Domain context
+ *
+ * Immutable state:
+ * @domain: The domain of this context.
+ * @key: Key in cla_domain::ctxs hashtable.
+ * @node: Node in cla_domain::ctxs hashtable.
+ *
+ * Mutable, reference counted:
+ * @refcnt: All references to this context. Context is freed
+ * when this drops to zero.
+ *
+ * Mutable, protected by domain::lock:
+ * @queue_node: Node in cla_domain::queued_ctxs.
+ * @waitq: Faulting threads sleep until assignment.
+ *
+ * Mutable, protected by domain::lock, some reads outside the lock:
+ * @users: Users accessing this context. Context is killed when
+ * this drops to zero.
+ *
+ * Mutable, written by domain reassignment, device work, and CPU hotplug:
+ * @regs: State for each device in domain.
+ * @asid: Pinned ASID of live context.
+ */
struct cla_ctx {
+ struct kref refcnt;
+ struct rhash_head node;
+
+ struct cla_domain *domain;
struct cla_ctx_key key;
+
+ refcount_t users;
+ struct list_head queue_node;
void **regs;
unsigned int asid;
+ wait_queue_head_t waitq;
};
extern struct xarray cla_domains;
@@ -142,7 +228,39 @@ static inline void cla_reg_write(struct cla_dev *dev, off_t reg, u64 val)
struct cla_domain *cla_dev_domain_get(struct cla_dev *dev);
int cla_domains_finalise(void);
+void cla_domains_sched_exit(void);
void cla_domains_free(void);
+int cla_domain_sched_init(struct cla_domain *domain);
+void cla_domain_sched_exit(struct cla_domain *domain);
+void cla_domain_set_broken(struct cla_domain *domain);
+struct cla_ctx *cla_domain_lookup_ctx(struct cla_domain *domain,
+ struct cla_ctx_key *key);
+int cla_domain_insert_ctx(struct cla_domain *domain, struct cla_ctx *ctx);
+void cla_domain_remove_ctx(struct cla_domain *domain, struct cla_ctx *ctx);
+void cla_domain_schedule_reassignment(struct cla_domain *domain);
+struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
+ struct cla_ctx_key *key);
+void cla_ctx_destroy(struct cla_domain *domain, struct cla_ctx_key *key);
+void cla_ctx_free(struct kref *ref);
+
+static inline void cla_ctx_get(struct cla_ctx *ctx)
+{
+ kref_get(&ctx->refcnt);
+}
+
+static inline void cla_ctx_put(struct cla_ctx *ctx)
+{
+ kref_put(&ctx->refcnt, cla_ctx_free);
+}
+
+static inline bool cla_ctx_is_dying(struct cla_ctx *ctx)
+{
+ return refcount_read(&ctx->users) == 0;
+}
+
+void cla_dev_switch_ctx_work(struct kthread_work *work);
+int cla_dev_switch_ctx(struct cla_dev *dev, struct cla_ctx *prev_ctx,
+ struct cla_ctx *next_ctx);
int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
int cla_op_reset(struct cla_dev *dev, unsigned int accid);
diff --git a/drivers/misc/arm-cla/cla-ctx.c b/drivers/misc/arm-cla/cla-ctx.c
new file mode 100644
index 000000000000..6a9c95769724
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-ctx.c
@@ -0,0 +1,146 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/err.h>
+#include <linux/file.h>
+#include <linux/gfp.h>
+#include <linux/sched/mm.h>
+#include <linux/slab.h>
+
+#include "arm-cla.h"
+
+/**
+ * cla_ctx_create - Create a context or increase its user count
+ * @domain: CLA domain
+ * @key: Context identifier
+ *
+ * Reuse an existing context for the same @key, or allocate and insert a new
+ * context into @domain. The context user count is incremented before returning.
+ *
+ * Return: the context on success, or an error pointer on failure
+ */
+struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
+ struct cla_ctx_key *key)
+{
+ struct cla_ctx *ctx;
+ int ret = -ENOMEM;
+
+ mutex_lock(&domain->lock);
+ ctx = cla_domain_lookup_ctx(domain, key);
+ if (ctx) {
+ refcount_inc(&ctx->users);
+ mutex_unlock(&domain->lock);
+ return ctx;
+ }
+
+ if (domain->broken) {
+ ret = -EIO;
+ goto err_unlock;
+ }
+
+ ctx = kzalloc_obj(*ctx, GFP_KERNEL_ACCOUNT);
+ if (!ctx)
+ goto err_unlock;
+
+ kref_init(&ctx->refcnt);
+ refcount_set(&ctx->users, 1);
+ ctx->domain = domain;
+ ctx->key = *key;
+ INIT_LIST_HEAD(&ctx->queue_node);
+ init_waitqueue_head(&ctx->waitq);
+
+ mmgrab(key->mm);
+ get_file(key->file);
+
+ /*
+ * Test if the mm or kvm is compatible. If cla_mtc_get_ctx() fails
+ * during context switch, it breaks the domain.
+ */
+ ret = cla_mtc_get_ctx(ctx);
+ if (ret)
+ goto err_free;
+ cla_mtc_put_ctx(ctx);
+
+ ret = cla_ctx_alloc_regs(domain, ctx);
+ if (ret)
+ goto err_free;
+
+ ret = cla_domain_insert_ctx(domain, ctx);
+ if (ret)
+ goto err_free_regs;
+ mutex_unlock(&domain->lock);
+
+ return ctx;
+
+err_free_regs:
+ cla_ctx_free_regs(domain, ctx);
+err_free:
+ mmdrop(key->mm);
+ fput(key->file);
+ kfree(ctx);
+err_unlock:
+ mutex_unlock(&domain->lock);
+ return ERR_PTR(ret);
+}
+
+/**
+ * cla_ctx_destroy - Drop a context user
+ * @domain: CLA domain
+ * @key: Context identifier
+ *
+ * When the final user is removed, wake waiters, remove the context from
+ * @domain, schedule deassignment if needed, and drop the user's reference.
+ */
+void cla_ctx_destroy(struct cla_domain *domain, struct cla_ctx_key *key)
+{
+ struct cla_ctx *ctx;
+
+ mutex_lock(&domain->lock);
+ ctx = cla_domain_lookup_ctx(domain, key);
+ WARN_ON(!ctx);
+
+ if (!refcount_dec_and_test(&ctx->users)) {
+ mutex_unlock(&domain->lock);
+ return;
+ }
+
+ /* Notify waiters that their context is dying. */
+ wake_up_all(&ctx->waitq);
+ cla_domain_remove_ctx(domain, ctx);
+ list_del_init(&ctx->queue_node);
+
+ /*
+ * Since we're holding the mmap_lock for writing, and reassignment work
+ * may be waiting to grab it for reading, we cannot wait for
+ * reassignment here. The context holds a reference to mm_count, so the
+ * mm_struct or pgd are not going away until cla_ctx_free().
+ * If the mm is exiting, the accelerators will now access memory through
+ * an empty pgd and fault silently.
+ *
+ * Assignment holds a reference until deassignment completes. If this
+ * context is being assigned, reassignment will notice that it is dying
+ * and schedule its deassignment once it re-takes the domain lock.
+ */
+ if (domain->assigned_ctx == ctx)
+ cla_domain_schedule_reassignment(domain);
+ mutex_unlock(&domain->lock);
+
+ /* Release initial reference from cla_ctx_create(). */
+ cla_ctx_put(ctx);
+}
+
+/**
+ * cla_ctx_free - Free a CLA domain context
+ * @ref: context reference counter
+ *
+ * Release resources held by a dying context after final reference is dropped.
+ */
+void cla_ctx_free(struct kref *ref)
+{
+ struct cla_ctx *ctx = container_of(ref, struct cla_ctx, refcnt);
+
+ WARN_ON(!cla_ctx_is_dying(ctx));
+ mmdrop(ctx->key.mm);
+ fput(ctx->key.file);
+ cla_ctx_free_regs(ctx->domain, ctx);
+ kfree(ctx);
+}
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 58f37787a59c..e1694f84e05d 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -186,7 +186,10 @@ static void cla_dev_reinit(struct cla_dev *dev)
if (WARN_ON(cla_op_reset_all(dev)) ||
WARN_ON(cla_mtc_clear(dev))) {
+ mutex_lock(&dev->domain->lock);
dev->broken = true;
+ cla_domain_set_broken(dev->domain);
+ mutex_unlock(&dev->domain->lock);
return;
}
@@ -198,6 +201,37 @@ static void cla_dev_reinit(struct cla_dev *dev)
cla_reg_write(dev, CLA_REG_LRESP, 0);
}
+static int cla_dev_worker_init(struct cla_dev *dev, int cpu)
+{
+ struct kthread_worker *worker;
+
+ worker = kthread_run_worker_on_cpu(cpu, 0, "cla-dev-worker/%u");
+ if (IS_ERR(worker))
+ return PTR_ERR(worker);
+
+ WARN_ON(dev->worker);
+ dev->worker = worker;
+
+ return 0;
+}
+
+static void cla_dev_worker_destroy(struct cla_dev *dev)
+{
+ if (!dev->worker)
+ return;
+
+ /*
+ * CPU offlining excludes submitters holding cpus_read_lock(). During
+ * module removal, submitters are stopped before removing the hotplug
+ * callbacks, which only takes the hotplug read lock.
+ */
+ kthread_destroy_worker(dev->worker);
+ dev->worker = NULL;
+
+ /* Clear work->worker before any later flush or reuse with a new worker. */
+ kthread_init_work(&dev->call.switch_ctx, cla_dev_switch_ctx_work);
+}
+
static int cla_dev_setup(unsigned int cpu)
{
int i;
@@ -252,8 +286,23 @@ static int cla_dev_setup(unsigned int cpu)
dev->accelerators);
}
+ ret = cla_dev_worker_init(dev, cpu);
+ if (ret)
+ goto err;
+
+ /* Restore context if domain has an assigned context. */
+ mutex_lock(&dev->domain->lock);
+ ret = cla_dev_switch_ctx(dev, NULL, dev->domain->assigned_ctx);
+ if (ret)
+ cla_domain_set_broken(dev->domain);
+ mutex_unlock(&dev->domain->lock);
+ if (ret)
+ goto err_worker;
+
dev->discovered = true;
return 0;
+err_worker:
+ cla_dev_worker_destroy(dev);
err:
cla_dev_reinit(dev);
return ret;
@@ -262,6 +311,7 @@ static int cla_dev_setup(unsigned int cpu)
static int cla_dev_teardown(unsigned int cpu)
{
struct cla_dev *dev;
+ int ret;
/*
* Careful what we return here, the teardown path isn't really allowed
@@ -271,6 +321,15 @@ static int cla_dev_teardown(unsigned int cpu)
if (!dev)
return 0;
+ cla_dev_worker_destroy(dev);
+
+ /* Save context if domain has an assigned context. */
+ mutex_lock(&dev->domain->lock);
+ ret = cla_dev_switch_ctx(dev, dev->domain->assigned_ctx, NULL);
+ if (ret)
+ cla_domain_set_broken(dev->domain);
+ mutex_unlock(&dev->domain->lock);
+
cla_dev_reinit(dev);
return 0;
@@ -314,6 +373,7 @@ static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
dev->cpu = cpu;
dev->dev = parent;
+ kthread_init_work(&dev->call.switch_ctx, cla_dev_switch_ctx_work);
/* Attempt to find device domain, or allocate a new one */
dev->domain = cla_dev_domain_get(dev);
@@ -441,6 +501,7 @@ static int __init cla_module_init(void)
return 0;
err_hardware:
+ cla_domains_sched_exit();
if (cla_cpuhp_state >= 0)
cpuhp_remove_state(cla_cpuhp_state);
cla_domains_free();
@@ -451,6 +512,7 @@ static int __init cla_module_init(void)
static void __exit cla_module_exit(void)
{
+ cla_domains_sched_exit();
cpuhp_remove_state(cla_cpuhp_state);
cla_domains_free();
platform_driver_unregister(&cla_driver);
diff --git a/drivers/misc/arm-cla/cla-regs.c b/drivers/misc/arm-cla/cla-regs.c
index 2452a7601990..d8ed74230e16 100644
--- a/drivers/misc/arm-cla/cla-regs.c
+++ b/drivers/misc/arm-cla/cla-regs.c
@@ -128,16 +128,23 @@ static int cla_regs_uninstall_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
if (ret)
return ret;
- for (i = 0; i < CLA_NUM_DATA_REGS; i++)
- regs->data[i] = cla_reg_read(dev, CLA_REG_DATA(i));
+ /*
+ * No need to save a dying context. Note that we're not holding the
+ * domain lock during this check and the context might be about to die,
+ * but it's only an optimisation.
+ */
+ if (!cla_ctx_is_dying(ctx)) {
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ regs->data[i] = cla_reg_read(dev, CLA_REG_DATA(i));
- cla_for_each_accid(dev, accid) {
- ret = cla_regs_save_accel(dev, accid, regs, ®state_off);
- if (ret)
- return ret;
- }
+ cla_for_each_accid(dev, accid) {
+ ret = cla_regs_save_accel(dev, accid, regs, ®state_off);
+ if (ret)
+ return ret;
+ }
- regs->accel_valid = true;
+ regs->accel_valid = true;
+ }
/*
* "If the accelerator is non idle when ENTERSR is launched, then a
@@ -218,7 +225,8 @@ int cla_regs_install(struct cla_dev *dev, struct cla_ctx *ctx,
*
* Uninstall a user context, and restore the kernel context if necessary.
*
- * PL0 context: Save the DATA and LRESP registers and accelerator state.
+ * PL0 context: Save the DATA registers and accelerator state unless
+ * cla_ctx_is_dying() is true, since a dying context will not be restored.
*
* When this completes, all accelerators are idle. If this fails, some
* accelerators may still be running.
diff --git a/drivers/misc/arm-cla/cla-sched.c b/drivers/misc/arm-cla/cla-sched.c
new file mode 100644
index 000000000000..6b5b782ac33e
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-sched.c
@@ -0,0 +1,473 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm CLA driver - domain scheduling
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/cpuhplock.h>
+#include <linux/rhashtable.h>
+
+#include "arm-cla.h"
+
+static struct cla_ctx invalid_ctx;
+
+static const struct rhashtable_params ctxs_rht_params = {
+ .head_offset = offsetof(struct cla_ctx, node),
+ .key_offset = offsetof(struct cla_ctx, key),
+ .key_len = sizeof(((struct cla_ctx *)0)->key),
+ .automatic_shrinking = true,
+};
+
+static int __cla_domain_switch_ctx(struct cla_domain *domain,
+ struct cla_ctx *prev_ctx,
+ struct cla_ctx *next_ctx)
+{
+ int ret = 0;
+ unsigned int i;
+ struct cla_dev *dev;
+ struct cla_call_on_cpu *call;
+
+ lockdep_assert_cpus_held();
+
+ if (!prev_ctx && !next_ctx)
+ return 0;
+
+ for (i = 0; i < domain->nr_devs; i++) {
+ dev = domain->devs[i];
+ call = &dev->call;
+
+ call->sched.prev_ctx = prev_ctx;
+ call->sched.next_ctx = next_ctx;
+ call->ret = 0;
+
+ if (dev->broken)
+ ret = -EIO;
+ else if (dev->worker)
+ kthread_queue_work(dev->worker, &call->switch_ctx);
+ }
+
+ for (i = 0; i < domain->nr_devs; i++) {
+ dev = domain->devs[i];
+ call = &dev->call;
+
+ kthread_flush_work(&call->switch_ctx);
+
+ /*
+ * kthread worker and kthread_flush_work() both take the worker
+ * lock to respectively write and read the current_work,
+ * providing the desired memory ordering for ret.
+ */
+ if (call->ret)
+ ret = call->ret;
+ }
+
+ return ret;
+}
+
+static int cla_domain_switch_ctx(struct cla_domain *domain,
+ struct cla_ctx *prev_ctx,
+ struct cla_ctx *next_ctx,
+ struct cla_ctx **assigned_ctx)
+{
+ int ret;
+
+ /* Remove prev_ctx from all devices in the domain */
+ *assigned_ctx = prev_ctx;
+ ret = __cla_domain_switch_ctx(domain, prev_ctx, NULL);
+ if (ret)
+ return ret;
+
+ *assigned_ctx = NULL;
+ if (prev_ctx)
+ cla_mtc_put_ctx(prev_ctx);
+ if (next_ctx) {
+ ret = cla_mtc_get_ctx(next_ctx);
+ if (ret)
+ return ret;
+ }
+
+ /* Install next_ctx into all devices in the domain */
+ *assigned_ctx = next_ctx;
+ return __cla_domain_switch_ctx(domain, NULL, next_ctx);
+}
+
+/*
+ * Run on the cla's local CPU, either in its worker or a CPU hotplug callback.
+ * Work submitters hold cpus_read_lock() until completion, excluding hotplug
+ * callbacks and keeping the worker bound. Hotplug callbacks hold domain->lock
+ * to protect the assigned context. Other devices may switch concurrently.
+ * On failure, mark the device broken; the caller handles domain failure.
+ */
+int cla_dev_switch_ctx(struct cla_dev *dev, struct cla_ctx *prev_ctx,
+ struct cla_ctx *next_ctx)
+{
+ unsigned int ctx_id = dev->pg_offset - dev->domain->pg_offset;
+ int ret;
+
+ if (WARN_ON(smp_processor_id() != dev->cpu)) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ /* Save state for outgoing ctx. */
+ if (prev_ctx) {
+ ret = cla_regs_uninstall(dev, prev_ctx, ctx_id);
+ if (ret) {
+ cla_err(dev, "failed to remove regs: %d\n", ret);
+ goto err;
+ }
+
+ ret = cla_mtc_uninstall(dev, prev_ctx);
+ if (ret) {
+ cla_err(dev, "failed to remove mm: %d\n", ret);
+ goto err;
+ }
+ }
+
+ /* Restore state for incoming ctx. */
+ if (next_ctx) {
+ ret = cla_mtc_install(dev, next_ctx);
+ if (ret) {
+ cla_err(dev, "failed to setup mm: %d\n", ret);
+ goto err;
+ }
+
+ ret = cla_regs_install(dev, next_ctx, ctx_id);
+ if (ret) {
+ cla_err(dev, "failed to setup regs: %d\n", ret);
+ goto err;
+ }
+ }
+
+ return 0;
+err:
+ dev->broken = true;
+ return ret;
+}
+
+void cla_dev_switch_ctx_work(struct kthread_work *work)
+{
+ struct cla_call_on_cpu *call =
+ container_of(work, struct cla_call_on_cpu, switch_ctx);
+ struct cla_dev *dev = container_of(call, struct cla_dev, call);
+
+ call->ret = cla_dev_switch_ctx(dev, call->sched.prev_ctx,
+ call->sched.next_ctx);
+}
+
+static inline struct cla_ctx *cla_domain_get_next_ctx(struct cla_domain *domain)
+{
+ return list_first_entry_or_null(&domain->queued_ctxs, struct cla_ctx,
+ queue_node);
+}
+
+static inline struct cla_ctx *cla_domain_get_prev_ctx(struct cla_domain *domain,
+ struct cla_ctx *next_ctx)
+{
+ if (domain->assigned_ctx && (next_ctx || cla_ctx_is_dying(domain->assigned_ctx)))
+ return domain->assigned_ctx;
+ return NULL;
+}
+
+static void cla_vma_zap_domain(struct vm_area_struct *vma, struct cla_domain *domain)
+{
+ unsigned long vma_start = vma->vm_pgoff;
+ unsigned long vma_end = vma_start + vma_pages(vma);
+ unsigned long domain_start = domain->pg_offset;
+ unsigned long domain_end = domain_start + domain->nr_devs;
+ unsigned long start = max(vma_start, domain_start);
+ unsigned long end = min(vma_end, domain_end);
+ unsigned long addr;
+ unsigned long size;
+
+ if (start >= end)
+ return;
+
+ addr = vma->vm_start + ((start - vma_start) << PAGE_SHIFT);
+ size = (end - start) << PAGE_SHIFT;
+ zap_special_vma_range(vma, addr, size);
+}
+
+static void cla_domain_zap(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ struct vm_area_struct *vma;
+ struct vma_iterator vmi;
+
+ /*
+ * If the context is dying it has already been unmapped, no need to zap
+ * it here.
+ */
+ if (!ctx || cla_ctx_is_dying(ctx))
+ return;
+
+ if (!mmget_not_zero(ctx->key.mm))
+ return;
+
+ /*
+ * Iterate over vmas in prev_ctx's mm, looking for vmas that map
+ * prev_ctx's file. Zap any portions of those vmas that map the domain.
+ */
+ mmap_read_lock(ctx->key.mm);
+ vma_iter_init(&vmi, ctx->key.mm, 0);
+ for_each_vma(vmi, vma) {
+ if (vma->vm_file != ctx->key.file)
+ continue;
+
+ cla_vma_zap_domain(vma, domain);
+ }
+ mmap_read_unlock(ctx->key.mm);
+ mmput(ctx->key.mm);
+}
+
+static void cla_domain_reassign(struct kthread_work *work)
+{
+ struct cla_ctx *assigned_ctx = NULL;
+ struct cla_ctx *prev_ctx = NULL;
+ struct cla_ctx *next_ctx = NULL;
+ struct cla_domain *domain;
+ int ret;
+
+ domain = container_of(work, struct cla_domain, reassign.work);
+
+ /*
+ * Exclude cpuhp until assigned_ctx reflects the hardware again. cpuhp
+ * callbacks use it to save/restore the local device and must not see
+ * invalid_ctx. Take before domain->lock, also used by cpuhp callbacks.
+ */
+ cpus_read_lock();
+ mutex_lock(&domain->lock);
+ if (domain->broken)
+ goto out_unlock;
+
+ /* Figure out prev_ctx and next_ctx. NULL indicates don't [un]assign. */
+ next_ctx = cla_domain_get_next_ctx(domain);
+ prev_ctx = cla_domain_get_prev_ctx(domain, next_ctx);
+
+ if (!prev_ctx && !next_ctx)
+ goto out_unlock;
+
+ /* prev_ctx already holds its assignment reference. */
+ if (next_ctx)
+ cla_ctx_get(next_ctx);
+
+ /*
+ * We need to grab mmap_lock, but can't do so while holding the domain
+ * lock (since cla_vma_fault() grabs the domain lock while holding
+ * mmap_lock). The queue may change and contexts may be killed by
+ * cla_ctx_destroy() while the lock is dropped.
+ *
+ * No user is allowed to access CLA while we are reassigning. Disable
+ * assigned_ctx before zapping so cla_vma_fault() won't re-map the
+ * domain during reassignment. The sentinel also defers broken-domain
+ * cleanup until we know which context hardware may still access.
+ */
+ WRITE_ONCE(domain->assigned_ctx, &invalid_ctx);
+ mutex_unlock(&domain->lock);
+
+ /* Prevent prev_ctx from accessing any device in the domain. */
+ cla_domain_zap(domain, prev_ctx);
+
+ /* cpus_read_lock() ensures workers remain cpu-bound and alive. */
+ ret = cla_domain_switch_ctx(domain, prev_ctx, next_ctx, &assigned_ctx);
+
+ mutex_lock(&domain->lock);
+ WRITE_ONCE(domain->assigned_ctx, assigned_ctx);
+
+ /* Also handle failure reported by another thread during the switch. */
+ if (ret || domain->broken) {
+ cla_domain_set_broken(domain);
+ goto out_unlock;
+ }
+
+ /*
+ * Remove next_ctx from the queue and wake up all waiters so that the
+ * fault handler can map the domain into the newly assigned process.
+ * assigned_ctx must have been updated prior to waking the waiters.
+ *
+ * If next_ctx was killed while we were assigning it, schedule another
+ * reassignment to deassign it.
+ */
+ if (next_ctx) {
+ if (cla_ctx_is_dying(next_ctx)) {
+ cla_domain_schedule_reassignment(domain);
+ goto out_unlock;
+ }
+ list_del_init(&next_ctx->queue_node);
+ wake_up_all(&next_ctx->waitq);
+ }
+
+ /* If there are more queued ctxs, schedule the next reassignment. */
+ if (!list_empty(&domain->queued_ctxs))
+ cla_domain_schedule_reassignment(domain);
+
+out_unlock:
+ mutex_unlock(&domain->lock);
+ cpus_read_unlock();
+
+ /* Deliberately put references outside of the locks. */
+ if (next_ctx && next_ctx != assigned_ctx)
+ cla_ctx_put(next_ctx);
+ if (prev_ctx && prev_ctx != assigned_ctx)
+ cla_ctx_put(prev_ctx);
+}
+
+/**
+ * cla_domain_sched_init - Initialize scheduling for a CLA domain
+ * @domain: CLA domain
+ *
+ * Initialize domain context table, scheduling queues, worker, and work items.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_domain_sched_init(struct cla_domain *domain)
+{
+ int ret;
+
+ INIT_LIST_HEAD(&domain->queued_ctxs);
+ WRITE_ONCE(domain->assigned_ctx, NULL);
+
+ ret = rhashtable_init(&domain->ctxs, &ctxs_rht_params);
+ if (ret)
+ return ret;
+
+ domain->worker = kthread_run_worker(0, "cla-domain-worker");
+ if (IS_ERR(domain->worker)) {
+ rhashtable_destroy(&domain->ctxs);
+ return PTR_ERR(domain->worker);
+ }
+ kthread_init_delayed_work(&domain->reassign, cla_domain_reassign);
+
+ return 0;
+}
+
+/**
+ * cla_domain_sched_exit - Tear down scheduling for a CLA domain
+ * @domain: CLA domain
+ *
+ * Drain final deassignment, destroy the domain worker, and destroy the context
+ * table. All context users must have exited before calling this.
+ */
+void cla_domain_sched_exit(struct cla_domain *domain)
+{
+ /*
+ * Wait for any in-flight assignment and prevent it from requeuing
+ * itself. Otherwise, it could move the final reassignment past the
+ * flush barrier, leaving an assignment reference behind.
+ */
+ kthread_cancel_delayed_work_sync(&domain->reassign);
+ WARN_ON(!list_empty(&domain->queued_ctxs));
+
+ /*
+ * All context users have exited, so this run can only deassign the
+ * remaining context and release its assignment reference. It cannot
+ * requeue itself, so destroying the worker also drains this final run.
+ */
+ kthread_mod_delayed_work(domain->worker, &domain->reassign, 0);
+ kthread_destroy_worker(domain->worker);
+ WARN_ON(READ_ONCE(domain->assigned_ctx));
+
+ rhashtable_destroy(&domain->ctxs);
+}
+
+/**
+ * cla_domain_set_broken - Mark a CLA domain as broken
+ * @domain: CLA domain
+ *
+ * Prevent further assignment and wake contexts waiting for assignment. Release
+ * the assignment reference while retaining resources that failed hardware may
+ * still access. The caller must hold the domain lock.
+ */
+void cla_domain_set_broken(struct cla_domain *domain)
+{
+ struct cla_ctx *ctx;
+
+ lockdep_assert_held(&domain->lock);
+ WRITE_ONCE(domain->broken, true);
+
+ /* Notify any waiter that their context isn't getting reassigned. */
+ list_for_each_entry(ctx, &domain->queued_ctxs, queue_node)
+ wake_up_all(&ctx->waitq);
+
+ /*
+ * During a switch, assigned_ctx is invalid_ctx. Ignore for now; we will
+ * be called again at the end of the switch once the real context is
+ * reinstalled in assigned_ctx.
+ */
+ ctx = domain->assigned_ctx;
+ if (ctx && ctx != &invalid_ctx) {
+ WRITE_ONCE(domain->assigned_ctx, NULL);
+ cla_mtc_leak_ctx(ctx);
+ cla_ctx_put(ctx);
+ }
+}
+
+/**
+ * cla_domain_lookup_ctx - Look up a context in a CLA domain
+ * @domain: CLA domain
+ * @key: Context identifier
+ *
+ * The caller must hold the domain lock.
+ *
+ * Return: the matching context, or %NULL if no context was found
+ */
+struct cla_ctx *cla_domain_lookup_ctx(struct cla_domain *domain,
+ struct cla_ctx_key *key)
+{
+ lockdep_assert_held(&domain->lock);
+ return rhashtable_lookup_fast(&domain->ctxs, key, ctxs_rht_params);
+}
+
+/**
+ * cla_domain_insert_ctx - Insert a context into a CLA domain
+ * @domain: CLA domain
+ * @ctx: context to insert
+ *
+ * The caller must hold the domain lock.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_domain_insert_ctx(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ lockdep_assert_held(&domain->lock);
+ return rhashtable_insert_fast(&domain->ctxs, &ctx->node, ctxs_rht_params);
+}
+
+/**
+ * cla_domain_remove_ctx - Remove a context from a CLA domain
+ * @domain: CLA domain
+ * @ctx: context to remove
+ *
+ * The caller must hold the domain lock.
+ */
+void cla_domain_remove_ctx(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ lockdep_assert_held(&domain->lock);
+ rhashtable_remove_fast(&domain->ctxs, &ctx->node, ctxs_rht_params);
+}
+
+/**
+ * cla_domain_schedule_reassignment - Schedule domain reassignment
+ * @domain: CLA domain
+ *
+ * The caller must hold the domain lock. If a live context is currently
+ * assigned, schedule with a delay (CLA_SLICE_MS), otherwise reassign
+ * immediately. Do nothing if the domain is broken or a switch is in progress;
+ * the running switch handles new requests before returning. If reassignment is
+ * already scheduled, the delay is modified.
+ */
+void cla_domain_schedule_reassignment(struct cla_domain *domain)
+{
+ struct cla_ctx *ctx = domain->assigned_ctx;
+ unsigned long delay_jfs = 0;
+
+ lockdep_assert_held(&domain->lock);
+
+ if (domain->broken || ctx == &invalid_ctx)
+ return;
+
+ if (ctx && !cla_ctx_is_dying(ctx))
+ delay_jfs = msecs_to_jiffies(CLA_SLICE_MS);
+
+ kthread_mod_delayed_work(domain->worker, &domain->reassign, delay_jfs);
+}
diff --git a/drivers/misc/arm-cla/cla-topology.c b/drivers/misc/arm-cla/cla-topology.c
index 5137a0e041e5..ed7acec70019 100644
--- a/drivers/misc/arm-cla/cla-topology.c
+++ b/drivers/misc/arm-cla/cla-topology.c
@@ -37,6 +37,8 @@ static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int fwi
if (!domain)
return ERR_PTR(-ENOMEM);
+ mutex_init(&domain->lock);
+
ret = xa_insert(&cla_domains, fwid, domain, GFP_KERNEL);
if (ret < 0)
goto err_free_domain;
@@ -49,8 +51,14 @@ static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int fwi
}
domain->devs[0] = dev;
+ ret = cla_domain_sched_init(domain);
+ if (ret)
+ goto err_free_devs;
+
return domain;
+err_free_devs:
+ kfree(domain->devs);
err_free_id:
xa_erase(&cla_domains, fwid);
err_free_domain:
@@ -149,6 +157,21 @@ int __init cla_domains_finalise(void)
return ret;
}
+/**
+ * cla_domains_sched_exit - Drain and stop all domain schedulers
+ *
+ * All context users must have exited. Keep domains and lookup tables alive for
+ * CPU hotplug callbacks until those callbacks have been removed.
+ */
+void cla_domains_sched_exit(void)
+{
+ unsigned long fwid;
+ struct cla_domain *domain;
+
+ xa_for_each(&cla_domains, fwid, domain)
+ cla_domain_sched_exit(domain);
+}
+
/**
* cla_domains_free - free CLA domains and lookup tables
*
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 09/15] misc/arm-cla: Add userspace interface
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (7 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 08/15] misc/arm-cla: Manage domain contexts Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 10/15] KVM: arm64: Introduce support to pin VMIDs Ryan Roberts
` (5 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
Expose CLA devices through a character device so userspace can enumerate
the available hardware and map accelerator register frames.
Provide ioctl interface to report device topology, CPU affinity, domain
membership, mmap offsets, architecture version and attached accelerator
masks, together with the IIDR, DEVARCH and REVIDR of each accelerator.
CLA registers can only be read from the CPU local to the device, while
enumeration may occur on any CPU. Validate the supported CLA
architecture version during device setup and cache the CLA and
accelerator identification registers for later ioctl queries.
Support shared read-write mmap of one or more CLA register pages. Create
a context for every domain covered by the mapping and resolve faults
only while that context owns the domain. Queue unassigned contexts with
the domain scheduler, drop mmap_lock while waiting for assignment and
retry the fault after the context is woken.
Co-developed-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
MAINTAINERS | 1 +
drivers/misc/arm-cla/Makefile | 3 +-
drivers/misc/arm-cla/arm-cla-regs.h | 2 +
drivers/misc/arm-cla/arm-cla.h | 14 +
drivers/misc/arm-cla/cla-init.c | 27 ++
drivers/misc/arm-cla/cla-regs.c | 10 +
drivers/misc/arm-cla/cla-user.c | 541 ++++++++++++++++++++++++++++
include/uapi/linux/arm-cla.h | 198 ++++++++++
8 files changed, 795 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/arm-cla/cla-user.c
create mode 100644 include/uapi/linux/arm-cla.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 98a25b3de9b4..f77d75e8a9a8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2216,6 +2216,7 @@ S: Supported
F: Documentation/devicetree/bindings/misc/arm,cla.yaml
F: Documentation/misc-devices/arm-cla.rst
F: drivers/misc/arm-cla/
+F: include/uapi/linux/arm-cla.h
ARM ETHOS-U NPU DRIVER
M: Rob Herring (Arm) <robh@kernel.org>
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index a92c3594e5c9..f848c608e8f1 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -7,6 +7,7 @@ arm-cla-y := \
cla-ops.o \
cla-regs.o \
cla-sched.o \
- cla-topology.o
+ cla-topology.o \
+ cla-user.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla-regs.h b/drivers/misc/arm-cla/arm-cla-regs.h
index d20a88780032..167e9ab64051 100644
--- a/drivers/misc/arm-cla/arm-cla-regs.h
+++ b/drivers/misc/arm-cla/arm-cla-regs.h
@@ -64,6 +64,8 @@
CLA_STATUS_EXCEPT | \
CLA_STATUS_SRMODE)
+#define CLA_AIDR_PREP(maj, min) ((ULL(maj) << 8) | ULL(min))
+
/* Standard accelerator registers */
#define CLA_REG_IIDR 0x0000
#define CLA_REG_DEVARCH 0x0001
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index a797330956d9..e047094dd49c 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -42,6 +42,12 @@ struct cla_call_on_cpu {
struct kthread_work switch_ctx;
};
+struct cla_accel_desc {
+ u64 iidr;
+ u64 devarch;
+ u64 revidr;
+};
+
/**
* struct cla_dev - CLA device
*
@@ -54,6 +60,8 @@ struct cla_call_on_cpu {
* @pg_offset: Mmap offset of this device.
* @iassizes: Combined regstate of all accels in bytes.
* @domain: The domain this CLA belongs to.
+ * @aidr: CLA Architecture ID.
+ * @accel_descs: Accelerator ID registers.
*
* Mutable, protected by CPU hotplug locking:
* @discovered: Discovered state is cached.
@@ -70,6 +78,8 @@ struct cla_dev {
unsigned long pg_offset;
unsigned long iassizes;
struct cla_domain *domain;
+ u64 aidr;
+ struct cla_accel_desc accel_descs[CLA_NUM_ACC];
bool discovered;
bool broken;
@@ -291,4 +301,8 @@ int cla_mtc_get_ctx(struct cla_ctx *ctx);
void cla_mtc_put_ctx(struct cla_ctx *ctx);
void cla_mtc_leak_ctx(struct cla_ctx *ctx);
+int cla_user_init(void);
+void cla_user_exit(void);
+u8 cla_user_accels(struct cla_dev *dev, struct file *file);
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index e1694f84e05d..1afe20aa877f 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -84,6 +84,7 @@ static int cla_reset_pmu(struct cla_dev *dev, unsigned int accid)
*/
static int cla_dev_discover_accel(struct cla_dev *dev, unsigned int accid)
{
+ struct cla_accel_desc *desc = &dev->accel_descs[accid];
unsigned long iassizes;
u64 iassize;
u64 acap;
@@ -102,6 +103,20 @@ static int cla_dev_discover_accel(struct cla_dev *dev, unsigned int accid)
return 1;
}
+ /*
+ * Cache some standard accelerator registers that user space may query
+ * via ioctl from a remote CPU.
+ */
+ ret = cla_op_regread(dev, accid, CLA_REG_IIDR, 1, &desc->iidr);
+ if (ret)
+ return ret;
+ ret = cla_op_regread(dev, accid, CLA_REG_DEVARCH, 1, &desc->devarch);
+ if (ret)
+ return ret;
+ ret = cla_op_regread(dev, accid, CLA_REG_REVIDR, 1, &desc->revidr);
+ if (ret)
+ return ret;
+
ret = cla_op_regread(dev, accid, CLA_REG_IASSIZE, 1, &iassize);
if (ret)
return ret;
@@ -248,6 +263,12 @@ static int cla_dev_setup(unsigned int cpu)
return -EINVAL;
if (!dev->discovered) {
+ dev->aidr = cla_reg_read(dev, CLA_REG_CLAAIDR);
+
+ /* Check for supported CLA architecture version. */
+ if (dev->aidr != CLA_AIDR_PREP(2, 0))
+ return -EPROTONOSUPPORT;
+
/* Discard partial results from an earlier, failed discovery. */
dev->accelerators = 0;
dev->iassizes = 0;
@@ -498,6 +519,11 @@ static int __init cla_module_init(void)
cla_cpuhp_state = ret;
}
+ /* Must be last fallible step; see cla_user_init(). */
+ ret = cla_user_init();
+ if (ret)
+ goto err_hardware;
+
return 0;
err_hardware:
@@ -512,6 +538,7 @@ static int __init cla_module_init(void)
static void __exit cla_module_exit(void)
{
+ cla_user_exit();
cla_domains_sched_exit();
cpuhp_remove_state(cla_cpuhp_state);
cla_domains_free();
diff --git a/drivers/misc/arm-cla/cla-regs.c b/drivers/misc/arm-cla/cla-regs.c
index d8ed74230e16..d48311d0ef92 100644
--- a/drivers/misc/arm-cla/cla-regs.c
+++ b/drivers/misc/arm-cla/cla-regs.c
@@ -162,6 +162,8 @@ static int cla_regs_install_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
{
int i;
int ret;
+ u64 pl0ctrl = 0;
+ u8 accelerators;
unsigned int accid;
size_t regstate_off = 0;
struct cla_regs_pl0 *regs = ctx->regs[ctx_id];
@@ -192,6 +194,14 @@ static int cla_regs_install_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
cla_reg_write(dev, CLA_REG_DATA(i), regs->data[i]);
cla_reg_write(dev, CLA_REG_LRESP, regs->lresp);
+ /* Restrict which accelerators this context can access at PL0. */
+ accelerators = cla_user_accels(dev, ctx->key.file);
+ cla_for_each_accid(dev, accid) {
+ if (accelerators & BIT(accid))
+ pl0ctrl |= CLA_PLxCTRL_PREP(accid, CLA_PLxCTRL_AVAIL);
+ }
+ cla_reg_write(dev, CLA_REG_PL0CTRL, pl0ctrl);
+
return 0;
}
diff --git a/drivers/misc/arm-cla/cla-user.c b/drivers/misc/arm-cla/cla-user.c
new file mode 100644
index 000000000000..41cd5e4a7d01
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-user.c
@@ -0,0 +1,541 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm CLA driver - userspace interface
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/cdev.h>
+#include <linux/cpu.h>
+#include <linux/device.h>
+#include <linux/fs.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include "arm-cla.h"
+#include <uapi/linux/arm-cla.h>
+
+static struct class *cla_class;
+static dev_t cla_devt;
+static struct cdev cla_cdev;
+static bool cla_cdev_added;
+static unsigned int cla_nr_minors;
+static unsigned int cla_nr_classes;
+static struct cla_accel_class **cla_accel_classes;
+
+struct cla_accel_class {
+ struct cla_accel_desc desc;
+ struct device dev;
+};
+
+static bool cla_accel_class_same(const struct cla_accel_desc *a,
+ const struct cla_accel_desc *b)
+{
+ u64 mask = CLA_IIDR_IMPLEMENTER | CLA_IIDR_PRODUCTID;
+
+ return (a->iidr & mask) == (b->iidr & mask);
+}
+
+u8 cla_user_accels(struct cla_dev *dev, struct file *file)
+{
+ struct cla_accel_class *aclass = file->private_data;
+ struct cla_accel_desc *desc = &aclass->desc;
+ unsigned int accid;
+ u8 mask = 0;
+
+ cla_for_each_accid(dev, accid) {
+ if (cla_accel_class_same(&dev->accel_descs[accid], desc))
+ mask |= BIT(accid);
+ }
+
+ return mask;
+}
+
+#define dev_nospec(idx) array_index_nospec(idx, cla_nr_devs)
+#define accel_nospec(idx) array_index_nospec(idx, CLA_NUM_ACC)
+
+#define cla_for_each_mapped_domain(_vma, _domain) \
+ for (unsigned long __pg = (_vma)->vm_pgoff; \
+ __pg < (_vma)->vm_pgoff + vma_pages(_vma) && \
+ ((_domain) = cla_lut_pg[dev_nospec(__pg)]->domain); \
+ __pg = (_domain)->pg_offset + (_domain)->nr_devs)
+
+static long cla_ioctl_validate_param(struct file *file,
+ struct arm_cla_param *param)
+{
+ u8 accelerators;
+ int accel_id;
+ int dev_id;
+
+ switch (param->param) {
+ case ARM_CLA_PARAM_DEV_NR:
+ if (param->index != 0)
+ return -EINVAL;
+ break;
+ case ARM_CLA_PARAM_DEV_CPU_ID:
+ case ARM_CLA_PARAM_DEV_DOMAIN_ID:
+ case ARM_CLA_PARAM_DEV_PGOFF:
+ case ARM_CLA_PARAM_DEV_AIDR:
+ case ARM_CLA_PARAM_DEV_ACCELS:
+ if (param->index >= cla_nr_devs)
+ return -EINVAL;
+ break;
+ case ARM_CLA_PARAM_ACCEL_IIDR:
+ case ARM_CLA_PARAM_ACCEL_DEVARCH:
+ case ARM_CLA_PARAM_ACCEL_REVIDR:
+ dev_id = ARM_CLA_PARAM_INDEX_DEV(param->index);
+ accel_id = ARM_CLA_PARAM_INDEX_ACCEL(param->index);
+ if (dev_id >= cla_nr_devs || accel_id >= CLA_NUM_ACC)
+ return -EINVAL;
+ dev_id = dev_nospec(dev_id);
+ accelerators = cla_user_accels(cla_lut_pg[dev_id], file);
+ if ((accelerators & BIT(accel_id)) == 0)
+ return -ENODEV;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static long cla_ioctl_get_param(struct file *file, unsigned long arg)
+{
+ struct arm_cla_param __user *uparam = (void __user *)arg;
+ struct arm_cla_param param;
+ int accel_id;
+ int dev_id;
+ int ret;
+
+ if (copy_from_user(¶m, uparam, sizeof(param)))
+ return -EFAULT;
+
+ ret = cla_ioctl_validate_param(file, ¶m);
+ if (ret)
+ return ret;
+
+ dev_id = dev_nospec(ARM_CLA_PARAM_INDEX_DEV(param.index));
+ accel_id = accel_nospec(ARM_CLA_PARAM_INDEX_ACCEL(param.index));
+
+ switch (param.param) {
+ case ARM_CLA_PARAM_DEV_NR:
+ param.value = cla_nr_devs;
+ break;
+ case ARM_CLA_PARAM_DEV_CPU_ID:
+ param.value = cla_lut_pg[dev_id]->cpu;
+ break;
+ case ARM_CLA_PARAM_DEV_DOMAIN_ID:
+ param.value = cla_lut_pg[dev_id]->domain->id;
+ break;
+ case ARM_CLA_PARAM_DEV_PGOFF:
+ param.value = cla_lut_pg[dev_id]->pg_offset;
+ break;
+ case ARM_CLA_PARAM_DEV_AIDR:
+ param.value = cla_lut_pg[dev_id]->aidr;
+ break;
+ case ARM_CLA_PARAM_DEV_ACCELS:
+ param.value = cla_user_accels(cla_lut_pg[dev_id], file);
+ break;
+ case ARM_CLA_PARAM_ACCEL_IIDR:
+ param.value = cla_lut_pg[dev_id]->accel_descs[accel_id].iidr;
+ break;
+ case ARM_CLA_PARAM_ACCEL_DEVARCH:
+ param.value = cla_lut_pg[dev_id]->accel_descs[accel_id].devarch;
+ break;
+ case ARM_CLA_PARAM_ACCEL_REVIDR:
+ param.value = cla_lut_pg[dev_id]->accel_descs[accel_id].revidr;
+ break;
+ }
+
+ if (copy_to_user(uparam, ¶m, sizeof(param)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static void cla_vma_open(struct vm_area_struct *vma)
+{
+ /*
+ * A vma previously created with cla_file_mmap() has been duplicated
+ * within the same mm (most likely due to mremap). While this could also
+ * be called for duplication into a new mm (via fork), we set
+ * VM_DONTCOPY on the original mmap, so this will never happen. So the
+ * contexts covered by this new vma already exist.
+ */
+ struct cla_ctx_key key = CLA_CTX_KEY_PL0(vma->vm_mm, vma->vm_file);
+ struct cla_domain *domain;
+
+ cla_for_each_mapped_domain(vma, domain)
+ cla_ctx_create(domain, &key);
+}
+
+static void cla_vma_close(struct vm_area_struct *vma)
+{
+ struct cla_ctx_key key = CLA_CTX_KEY_PL0(vma->vm_mm, vma->vm_file);
+ struct cla_domain *domain;
+
+ /* On munmap() or exit_mmap(), kill the context. */
+ cla_for_each_mapped_domain(vma, domain)
+ cla_ctx_destroy(domain, &key);
+}
+
+static vm_fault_t cla_vma_fault(struct vm_fault *vmf)
+{
+ struct vm_area_struct *vma = vmf->vma;
+ struct cla_ctx_key key = CLA_CTX_KEY_PL0(vma->vm_mm, vma->vm_file);
+ struct cla_domain *domain;
+ struct cla_dev *dev;
+ struct cla_ctx *ctx;
+ unsigned long pg;
+
+ /*
+ * Reassignment needs mmap_lock, so we cannot wait while holding it.
+ * The private futex fault-in path can reach here without allowing
+ * mmap_lock to be dropped. Reject faults that do not allow retry.
+ */
+ if (!(vmf->flags & FAULT_FLAG_ALLOW_RETRY))
+ return VM_FAULT_SIGBUS;
+
+ pg = vma->vm_pgoff + ((vmf->address - vma->vm_start) >> PAGE_SHIFT);
+ dev = cla_lut_pg[pg];
+ domain = dev->domain;
+
+ mutex_lock(&domain->lock);
+ ctx = cla_domain_lookup_ctx(domain, &key);
+ if (WARN_ON(!ctx) || cla_ctx_is_dying(ctx) || domain->broken) {
+ mutex_unlock(&domain->lock);
+ return domain->broken ? VM_FAULT_SIGBUS : VM_FAULT_SIGSEGV;
+ }
+
+ /* If our ctx is the assigned one, map the device into memory. */
+ if (domain->assigned_ctx == ctx) {
+ vm_fault_t ret;
+
+ ret = vmf_insert_pfn(vma, vmf->address, dev->pfn);
+ mutex_unlock(&domain->lock);
+ return ret;
+ }
+
+ /* Enqueue if not already, starting current assignee's time slice. */
+ if (list_empty(&ctx->queue_node)) {
+ bool was_empty = list_empty(&domain->queued_ctxs);
+
+ list_add_tail(&ctx->queue_node, &domain->queued_ctxs);
+ if (was_empty)
+ cla_domain_schedule_reassignment(domain);
+ }
+
+ mutex_unlock(&domain->lock);
+
+ /* If waiting is not permitted, return indicating we need to retry. */
+ if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT)
+ return VM_FAULT_RETRY;
+
+ /*
+ * Drop the fault lock prior to sleeping and return VM_FAULT_RETRY when
+ * we wake up to rerun the fault. We must get the ctx while sleeping to
+ * prevent it from being freed while we are asleep on the contained wait
+ * queue.
+ */
+ cla_ctx_get(ctx);
+ release_fault_lock(vmf);
+ wait_event_interruptible(ctx->waitq,
+ READ_ONCE(domain->assigned_ctx) == ctx ||
+ cla_ctx_is_dying(ctx) ||
+ READ_ONCE(domain->broken));
+ cla_ctx_put(ctx);
+ return VM_FAULT_RETRY;
+}
+
+static int cla_vma_may_split(struct vm_area_struct *vma, unsigned long addr)
+{
+ /*
+ * Forbid splitting cla mappings to prevent refcount leaks.
+ * cla_file_mmap()/cla_vma_open() track a context per domain mapped by
+ * the vma. If there were fewer domains mapped by the vma at
+ * cla_vma_close() then contexts would get leaked.
+ */
+ return -EINVAL;
+}
+
+static const struct vm_operations_struct cla_vma_ops = {
+ .open = cla_vma_open,
+ .close = cla_vma_close,
+ .fault = cla_vma_fault,
+ .may_split = cla_vma_may_split,
+};
+
+static int cla_file_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ unsigned long nr_pages = vma_pages(vma);
+ struct cla_ctx_key key = CLA_CTX_KEY_PL0(vma->vm_mm, vma->vm_file);
+ struct cla_domain *rollback;
+ struct cla_domain *domain;
+ struct cla_ctx *ctx;
+
+ /* Ensure the requested mapping is within range. */
+ if (!nr_pages)
+ return -EINVAL;
+ if (vma->vm_pgoff >= cla_nr_devs)
+ return -EINVAL;
+ if (nr_pages > cla_nr_devs - vma->vm_pgoff)
+ return -EINVAL;
+
+ /* Only allow shared RW mappings. Nothing else makes sense for CLA. */
+ if (!(vma->vm_flags & VM_SHARED))
+ return -EINVAL;
+ if (!(vma->vm_flags & VM_READ))
+ return -EINVAL;
+ if (!(vma->vm_flags & VM_WRITE))
+ return -EINVAL;
+ if (vma->vm_flags & VM_EXEC)
+ return -EINVAL;
+
+ /*
+ * Iterate over each domain covered by the vma and get-or-alloc its
+ * context. If any fails, we need to rollback with a put.
+ */
+ cla_for_each_mapped_domain(vma, domain) {
+ ctx = cla_ctx_create(domain, &key);
+ if (IS_ERR(ctx)) {
+ cla_for_each_mapped_domain(vma, rollback) {
+ if (rollback == domain)
+ return PTR_ERR(ctx);
+ cla_ctx_destroy(rollback, &key);
+ }
+ }
+ }
+
+ vm_flags_mod(vma, VM_DONTCOPY | VM_DONTDUMP | VM_DONTEXPAND |
+ VM_IO | VM_PFNMAP, VM_MAYEXEC);
+ vma->vm_page_prot = pgprot_device(vma->vm_page_prot);
+ vma->vm_ops = &cla_vma_ops;
+
+ return 0;
+}
+
+static long cla_file_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ switch (cmd) {
+ case ARM_CLA_IOCTL_GET_PARAM:
+ return cla_ioctl_get_param(file, arg);
+ default:
+ return -ENOTTY;
+ }
+}
+
+static int cla_file_open(struct inode *inode, struct file *file)
+{
+ unsigned int index = iminor(inode) - MINOR(cla_devt);
+
+ if (index >= cla_nr_classes)
+ return -ENODEV;
+
+ file->private_data = cla_accel_classes[index];
+
+ return 0;
+}
+
+static const struct file_operations cla_fops = {
+ .owner = THIS_MODULE,
+ .open = cla_file_open,
+ .mmap = cla_file_mmap,
+ .unlocked_ioctl = cla_file_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
+};
+
+static char *cla_devnode(const struct device *dev, umode_t *mode)
+{
+ if (mode)
+ *mode = 0666;
+ return kasprintf(GFP_KERNEL, "cla/%s", dev_name(dev));
+}
+
+static void cla_accel_class_release(struct device *dev)
+{
+ kfree(container_of(dev, struct cla_accel_class, dev));
+}
+
+static bool __init cla_accel_class_seen(unsigned int dev_id, unsigned int accid)
+{
+ const struct cla_accel_desc *desc = &cla_lut_pg[dev_id]->accel_descs[accid];
+ unsigned int i, j;
+
+ for (i = 0; i <= dev_id; i++) {
+ struct cla_dev *dev = cla_lut_pg[i];
+
+ cla_for_each_accid(dev, j) {
+ /* Only compare accelerator slots preceding this one. */
+ if (i == dev_id && j == accid)
+ return false;
+ if (cla_accel_class_same(desc, &dev->accel_descs[j]))
+ return true;
+ }
+ }
+
+ return false;
+}
+
+static int __init cla_accel_class_add(const struct cla_accel_desc *desc)
+{
+ struct cla_accel_class *aclass;
+ unsigned int i;
+ int ret;
+
+ /* Check if an appropriate accel class was already created. */
+ for (i = 0; i < cla_nr_classes; i++) {
+ if (cla_accel_class_same(desc, &cla_accel_classes[i]->desc))
+ return 0;
+ }
+
+ aclass = kzalloc_obj(*aclass);
+ if (!aclass)
+ return -ENOMEM;
+
+ aclass->desc = *desc;
+ device_initialize(&aclass->dev);
+ aclass->dev.class = cla_class;
+ aclass->dev.devt = cla_devt + cla_nr_classes;
+ aclass->dev.release = cla_accel_class_release;
+
+ ret = dev_set_name(&aclass->dev, "%03lx:%03lx",
+ FIELD_GET(CLA_IIDR_IMPLEMENTER, desc->iidr),
+ FIELD_GET(CLA_IIDR_PRODUCTID, desc->iidr));
+ if (ret)
+ goto err_put;
+
+ /* The node cannot be opened until cla_user_init() adds the cdev. */
+ ret = device_add(&aclass->dev);
+ if (ret)
+ goto err_put;
+
+ cla_accel_classes[cla_nr_classes++] = aclass;
+ return 0;
+
+err_put:
+ put_device(&aclass->dev);
+ return ret;
+}
+
+/**
+ * cla_user_init - Initialize and publish the CLA userspace interface
+ *
+ * Create a device for each unique accelerator class, then publish all class
+ * minors through a single cdev. Device discovery must have completed before
+ * calling this function.
+ *
+ * This must be the last fallible step of module initialization: cdev_add()
+ * cannot fail after making the devices openable. On failure, all resources
+ * allocated here are released without allowing any opens.
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int __init cla_user_init(void)
+{
+ unsigned int dev_id, accid;
+ int ret;
+
+ if (!cla_nr_devs)
+ return -EINVAL;
+
+ /* Count unique classes before allocating the minor range and lookup. */
+ for (dev_id = 0; dev_id < cla_nr_devs; dev_id++) {
+ struct cla_dev *dev = cla_lut_pg[dev_id];
+
+ cla_for_each_accid(dev, accid) {
+ if (cla_accel_class_seen(dev_id, accid))
+ continue;
+ if (++cla_nr_minors > MINORMASK + 1)
+ return -EINVAL;
+ }
+ }
+
+ if (!cla_nr_minors)
+ return 0;
+
+ cla_class = class_create("arm-cla");
+ if (IS_ERR(cla_class))
+ return PTR_ERR(cla_class);
+ cla_class->devnode = cla_devnode;
+
+ ret = alloc_chrdev_region(&cla_devt, 0, cla_nr_minors, KBUILD_MODNAME);
+ if (ret)
+ goto err_class_destroy;
+
+ cla_accel_classes = kvzalloc_objs(*cla_accel_classes, cla_nr_minors);
+ if (!cla_accel_classes) {
+ ret = -ENOMEM;
+ goto err_user_exit;
+ }
+
+ /* Prepare every unique accelerator class before publishing the cdev. */
+ for (dev_id = 0; dev_id < cla_nr_devs; dev_id++) {
+ struct cla_dev *dev = cla_lut_pg[dev_id];
+
+ cla_for_each_accid(dev, accid) {
+ ret = cla_accel_class_add(&dev->accel_descs[accid]);
+ if (ret)
+ goto err_user_exit;
+ }
+ }
+
+ /*
+ * Once cdev_add() succeeds, user space can open the device file. Module
+ * initialization must not fail after this point because the cdev
+ * depends on the module.
+ */
+ cdev_init(&cla_cdev, &cla_fops);
+ cla_cdev.owner = THIS_MODULE;
+ ret = cdev_add(&cla_cdev, cla_devt, cla_nr_classes);
+ if (ret) {
+ kobject_put(&cla_cdev.kobj);
+ goto err_user_exit;
+ }
+ cla_cdev_added = true;
+
+ return 0;
+
+err_user_exit:
+ cla_user_exit();
+ return ret;
+err_class_destroy:
+ class_destroy(cla_class);
+ return ret;
+}
+
+/**
+ * cla_user_exit - Remove the CLA userspace interface
+ *
+ * Remove the cdev, unregister the accelerator class devices, and release the
+ * minor lookup, device numbers and device class.
+ *
+ * Called during module exit or by cla_user_init() to unwind a failure before
+ * cdev publication. No open files may remain: the module references held by
+ * open files ensure this during normal module removal.
+ */
+void cla_user_exit(void)
+{
+ unsigned int i;
+
+ if (!cla_nr_minors)
+ return;
+
+ if (cla_cdev_added) {
+ cdev_del(&cla_cdev);
+ cla_cdev_added = false;
+ }
+
+ for (i = 0; i < cla_nr_classes; i++)
+ device_unregister(&cla_accel_classes[i]->dev);
+
+ kvfree(cla_accel_classes);
+ cla_accel_classes = NULL;
+ cla_nr_classes = 0;
+ unregister_chrdev_region(cla_devt, cla_nr_minors);
+ cla_nr_minors = 0;
+ class_destroy(cla_class);
+}
diff --git a/include/uapi/linux/arm-cla.h b/include/uapi/linux/arm-cla.h
new file mode 100644
index 000000000000..f65f1b760c34
--- /dev/null
+++ b/include/uapi/linux/arm-cla.h
@@ -0,0 +1,198 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * UAPI definitions for the CLA character device.
+ */
+
+#ifndef _UAPI_LINUX_CLA_H
+#define _UAPI_LINUX_CLA_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+#define ARM_CLA_IOC_MAGIC 'C'
+
+/**
+ * define ARM_CLA_IOCTL_GET_PARAM - Get the value of a parameter.
+ *
+ * ioctl command whose argument is a pointer to &struct arm_cla_param.
+ * &arm_cla_param->param and &arm_cla_param->index are input parameters.
+ * &arm_cla_param->value is an output parameter.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - Unrecognised param or param does not support GET
+ * * Return code as defined for param
+ */
+#define ARM_CLA_IOCTL_GET_PARAM \
+ _IOWR(ARM_CLA_IOC_MAGIC, 0x00, struct arm_cla_param)
+
+/**
+ * define ARM_CLA_PARAM_DEV_NR - Number of attached CLA devices (RO).
+ *
+ * &arm_cla_param->index must be 0.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - index not 0
+ */
+#define ARM_CLA_PARAM_DEV_NR 1
+
+/**
+ * define ARM_CLA_PARAM_DEV_CPU_ID - CPU to which the CLA is attached (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range [0, DEV_NR).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_CPU_ID 2
+
+/**
+ * define ARM_CLA_PARAM_DEV_DOMAIN_ID - Domain to which the CLA belongs (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range [0, DEV_NR).
+ *
+ * A CLA domain contains a set of CLA devices whose accelerators can communicate
+ * with each other. All CLA devices within a CLA domain are atomically assigned
+ * to a single context at a time.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_DOMAIN_ID 3
+
+/**
+ * define ARM_CLA_PARAM_DEV_PGOFF - CLA's ``mmap()`` page offset (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range [0, DEV_NR).
+ *
+ * The page offset at which the selected CLA device's register page is exposed
+ * through ``mmap()``.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_PGOFF 4
+
+/**
+ * define ARM_CLA_PARAM_DEV_AIDR - CLA Architecture Identification Register
+ * (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range [0, DEV_NR).
+ *
+ * The CLA Architecture Identification Register is used to obtain version
+ * information of the CLA Programming model.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_AIDR 5
+
+/**
+ * define ARM_CLA_PARAM_DEV_ACCELS - CLA accelerators in opened class (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range [0, DEV_NR).
+ *
+ * A bitmask of accelerator slots on the selected CLA device that belong to the
+ * class of the opened character device. Class membership is determined by
+ * matching IIDR.IMPLEMENTER and IIDR.PRODUCTID.
+ *
+ * Each bit corresponds to an accel_id in the range [0, 8). The mask is zero if
+ * the selected CLA has no accelerators belonging to the opened class.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_ACCELS 6
+
+/**
+ * define ARM_CLA_PARAM_ACCEL_IIDR - Accelerator IIDR register (RO).
+ *
+ * &arm_cla_param->index is ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id not in range
+ * * %-EINVAL - accel_id not in range
+ * * %-ENODEV - accel_id not attached or does not belong to opened class
+ */
+#define ARM_CLA_PARAM_ACCEL_IIDR 7
+
+/**
+ * define ARM_CLA_PARAM_ACCEL_DEVARCH - Accelerator DEVARCH register (RO).
+ *
+ * &arm_cla_param->index is ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id not in range
+ * * %-EINVAL - accel_id not in range
+ * * %-ENODEV - accel_id not attached or does not belong to opened class
+ */
+#define ARM_CLA_PARAM_ACCEL_DEVARCH 8
+
+/**
+ * define ARM_CLA_PARAM_ACCEL_REVIDR - Accelerator REVIDR register (RO).
+ *
+ * &arm_cla_param->index is ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id not in range
+ * * %-EINVAL - accel_id not in range
+ * * %-ENODEV - accel_id not attached or does not belong to opened class
+ */
+#define ARM_CLA_PARAM_ACCEL_REVIDR 9
+
+/**
+ * ARM_CLA_PARAM_ACCEL_INDEX() - encode index for specific accelerator.
+ * @accel_id: accelerator id in the range [0, 8).
+ * @dev_id: CLA device id in the range [0, DEV_NR).
+ *
+ * Return: encoded index for use in &arm_cla_param->index.
+ */
+#define ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id) \
+ (((accel_id) << 24) | (dev_id))
+
+/**
+ * ARM_CLA_PARAM_INDEX_ACCEL() - extract accel_id from index.
+ * @index: Encoded index as returned by ARM_CLA_PARAM_ACCEL_INDEX()
+ *
+ * Return: extracted accel_id.
+ */
+#define ARM_CLA_PARAM_INDEX_ACCEL(index) (((index) >> 24) & 0xff)
+
+/**
+ * ARM_CLA_PARAM_INDEX_DEV() - extract dev_id from index.
+ * @index: Encoded index as returned by ARM_CLA_PARAM_ACCEL_INDEX()
+ *
+ * Return: extracted dev_id.
+ */
+#define ARM_CLA_PARAM_INDEX_DEV(index) ((index) & 0xffffff)
+
+/**
+ * struct arm_cla_param - Get/Set CLA parameters.
+ */
+struct arm_cla_param {
+ /**
+ * @param: Param selector - One of the ARM_CLA_PARAM_* values.
+ */
+ __u64 param;
+
+ /**
+ * @index: Index for params that have multiple instances.
+ */
+ __u64 index;
+
+ /**
+ * @value: Param value.
+ */
+ __u64 value;
+};
+
+#endif /* _UAPI_LINUX_CLA_H */
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 10/15] KVM: arm64: Introduce support to pin VMIDs
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (8 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 09/15] misc/arm-cla: Add userspace interface Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 11/15] KVM: arm64: Export stage-2 MMU config Ryan Roberts
` (4 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc, Shameer Kolothum
From: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Introduce kvm_arm_pinned_vmid_get() and kvm_arm_pinned_vmid_put(), to pin
a VMID associated with a KVM instance. This will guarantee that VMID
remains the same after a rollover.
This will allow some devices, such as ones behind an SMMUv3, to have
their TLB entries tagged with the KVM VMID. In some systems, TLB
maintenance instructions from the CPU to the Inner Shareable domain can
invalidate those entries.
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 3 ++
arch/arm64/kvm/vmid.c | 77 ++++++++++++++++++++++++++++---
2 files changed, 73 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index cd9b9d2462f9..30d0082b15ec 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -148,6 +148,7 @@ int topup_hyp_memcache(struct kvm_hyp_memcache *mc, unsigned long min_pages);
struct kvm_vmid {
atomic64_t id;
+ refcount_t pinned;
};
struct kvm_s2_mmu {
@@ -1359,6 +1360,8 @@ int __init kvm_arm_vmid_alloc_init(void);
void __init kvm_arm_vmid_alloc_free(void);
void kvm_arm_vmid_update(struct kvm_vmid *kvm_vmid);
void kvm_arm_vmid_clear_active(void);
+int kvm_arm_pinned_vmid_get(struct kvm *kvm);
+void kvm_arm_pinned_vmid_put(struct kvm *kvm);
static inline void kvm_arm_pvtime_vcpu_init(struct kvm_vcpu_arch *vcpu_arch)
{
diff --git a/arch/arm64/kvm/vmid.c b/arch/arm64/kvm/vmid.c
index 7fe8ba1a2851..322279f59e88 100644
--- a/arch/arm64/kvm/vmid.c
+++ b/arch/arm64/kvm/vmid.c
@@ -25,6 +25,10 @@ static unsigned long *vmid_map;
static DEFINE_PER_CPU(atomic64_t, active_vmids);
static DEFINE_PER_CPU(u64, reserved_vmids);
+static unsigned long max_pinned_vmids;
+static unsigned long nr_pinned_vmids;
+static unsigned long *pinned_vmid_map;
+
#define VMID_MASK (~GENMASK(kvm_arm_vmid_bits - 1, 0))
#define VMID_FIRST_VERSION (1UL << kvm_arm_vmid_bits)
@@ -47,7 +51,7 @@ static void flush_context(void)
int cpu;
u64 vmid;
- bitmap_zero(vmid_map, NUM_USER_VMIDS);
+ bitmap_copy(vmid_map, pinned_vmid_map, NUM_USER_VMIDS);
for_each_possible_cpu(cpu) {
vmid = atomic64_xchg_relaxed(&per_cpu(active_vmids, cpu), 0);
@@ -98,12 +102,14 @@ static u64 new_vmid(struct kvm_vmid *kvm_vmid)
if (vmid != 0) {
u64 newvmid = generation | (vmid & ~VMID_MASK);
- if (check_update_reserved_vmid(vmid, newvmid)) {
- atomic64_set(&kvm_vmid->id, newvmid);
- return newvmid;
- }
-
- if (!__test_and_set_bit(vmid2idx(vmid), vmid_map)) {
+ /*
+ * If it is pinned, we can keep using it. Note that reserved
+ * takes priority, because even if it is also pinned, we need to
+ * update the generation into the reserved_vmids.
+ */
+ if (check_update_reserved_vmid(vmid, newvmid) ||
+ refcount_read(&kvm_vmid->pinned) ||
+ !__test_and_set_bit(vmid2idx(vmid), vmid_map)) {
atomic64_set(&kvm_vmid->id, newvmid);
return newvmid;
}
@@ -169,6 +175,49 @@ void kvm_arm_vmid_update(struct kvm_vmid *kvm_vmid)
raw_spin_unlock_irqrestore(&cpu_vmid_lock, flags);
}
+int kvm_arm_pinned_vmid_get(struct kvm *kvm)
+{
+ struct kvm_vmid *kvm_vmid;
+ u64 vmid;
+
+ kvm_vmid = &kvm->arch.mmu.vmid;
+
+ guard(raw_spinlock_irqsave)(&cpu_vmid_lock);
+ vmid = atomic64_read(&kvm_vmid->id);
+
+ if (refcount_inc_not_zero(&kvm_vmid->pinned))
+ return (vmid & ~VMID_MASK);
+
+ if (nr_pinned_vmids >= max_pinned_vmids)
+ return -EINVAL;
+
+ /*
+ * If we went through one or more rollover since that VMID was
+ * used, make sure it is still valid, or generate a new one.
+ */
+ if (!vmid_gen_match(vmid))
+ vmid = new_vmid(kvm_vmid);
+
+ nr_pinned_vmids++;
+ __set_bit(vmid2idx(vmid), pinned_vmid_map);
+ refcount_set(&kvm_vmid->pinned, 1);
+ return (vmid & ~VMID_MASK);
+}
+
+void kvm_arm_pinned_vmid_put(struct kvm *kvm)
+{
+ struct kvm_vmid *kvm_vmid;
+ u64 vmid;
+
+ kvm_vmid = &kvm->arch.mmu.vmid;
+ vmid = atomic64_read(&kvm_vmid->id);
+ guard(raw_spinlock_irqsave)(&cpu_vmid_lock);
+ if (refcount_dec_and_test(&kvm_vmid->pinned)) {
+ __clear_bit(vmid2idx(vmid), pinned_vmid_map);
+ nr_pinned_vmids--;
+ }
+}
+
/*
* Initialize the VMID allocator
*/
@@ -186,10 +235,24 @@ int __init kvm_arm_vmid_alloc_init(void)
if (!vmid_map)
return -ENOMEM;
+ pinned_vmid_map = bitmap_zalloc(NUM_USER_VMIDS, GFP_KERNEL);
+ if (!pinned_vmid_map) {
+ bitmap_free(vmid_map);
+ return -ENOMEM;
+ }
+ nr_pinned_vmids = 0;
+
+ /*
+ * Ensure we have at least one empty slot available after rollover when
+ * the maximum number of VMIDs are pinned. VMID#0 is reserved.
+ */
+ max_pinned_vmids = NUM_USER_VMIDS - num_possible_cpus() - 2;
+
return 0;
}
void __init kvm_arm_vmid_alloc_free(void)
{
+ bitmap_free(pinned_vmid_map);
bitmap_free(vmid_map);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 11/15] KVM: arm64: Export stage-2 MMU config
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (9 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 10/15] KVM: arm64: Introduce support to pin VMIDs Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 12/15] vfio: Support VFIO devices that access the CPU MMU Ryan Roberts
` (3 subsequent siblings)
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Provide an API to drivers that want to share the stage 2 configuration
of a VM with their device. At the moment it is rather limited:
* No invalidation notifiers. The TLBs on the device side, if any,
receive CPU TLB invalidations.
* No remote page fault support. All paging still takes the normal data
abort route on the CPU. Options are pinning all guest memory or
device-specific mechanism for reporting pending faults to the guest.
* No nested VMs. Although we could perhaps support assigning devices to
the hyp context of a nested VM, it seems performative.
So for the moment this API works for devices that share the CPU MMU, but
the SMMU driver could use it with DVM.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
arch/arm64/include/asm/kvm_host.h | 11 ++++
arch/arm64/include/asm/kvm_mmu.h | 1 +
arch/arm64/kvm/mmu.c | 102 ++++++++++++++++++++++++++++++
arch/arm64/kvm/sys_regs.c | 14 +---
4 files changed, 117 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 30d0082b15ec..c0932b43f959 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -235,6 +235,14 @@ struct kvm_s2_mmu {
atomic_t refcnt;
};
+/* stage 2 MMU configuration, for drivers that speak VMSA */
+struct kvm_arch_guest_mmu {
+ u64 hcr;
+ u64 hcrx;
+ u64 vtcr;
+ u64 vttbr;
+};
+
struct kvm_arch_memory_slot {
};
@@ -1720,4 +1728,7 @@ static __always_inline enum fgt_group_id __fgt_reg_to_group_id(enum vcpu_sysreg
long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext);
+struct kvm_arch_guest_mmu *kvm_arch_get_guest_mmu(struct kvm *kvm);
+void kvm_arch_put_guest_mmu(struct kvm *kvm, struct kvm_arch_guest_mmu *mmu);
+
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6eae7e7e2a68..8a9d385fe14d 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -185,6 +185,7 @@ int kvm_handle_guest_sea(struct kvm_vcpu *vcpu);
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu);
phys_addr_t kvm_mmu_get_httbr(void);
+u64 kvm_mmu_get_hcr(struct kvm *kvm, u64 hcr);
phys_addr_t kvm_get_idmap_vector(void);
int __init kvm_mmu_init(u32 hyp_va_bits);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 2d44cd6a5aed..0bf1dff2a41f 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -2805,3 +2805,105 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
}
+
+#define HCR_MMU_BITS (HCR_VM | HCR_PTW | HCR_DC | HCR_TGE | HCR_RW | \
+ HCR_CD | HCR_ID | HCR_E2H | HCR_NV | HCR_NV1 | \
+ HCR_FWB | HCR_ATA | HCR_DCT)
+
+/* Update MMU-specific host bits of HCR_EL2 */
+u64 kvm_mmu_get_hcr(struct kvm *kvm, u64 hcr)
+{
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ WARN_ON(!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED,
+ &kvm->arch.flags));
+
+ hcr &= ~HCR_MMU_BITS;
+ hcr |= HCR_VM | HCR_PTW;
+
+ if (has_vhe() || has_hvhe())
+ hcr |= HCR_E2H;
+
+ if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+ hcr |= HCR_FWB;
+
+ if (kvm_has_mte(kvm))
+ hcr |= HCR_ATA;
+
+ if (!test_bit(KVM_ARM_VCPU_EL1_32BIT, kvm->arch.vcpu_features))
+ hcr |= HCR_RW;
+
+ return hcr;
+}
+
+/**
+ * kvm_arch_get_guest_mmu - Get a reference to the guest MMU config
+ * @kvm: KVM instance. Caller must hold a reference.
+ *
+ * Get the stage 2 MMU config of an EL1 VM. The returned object contains
+ * settings that do not change until the object is released with
+ * kvm_arch_put_guest_mmu(). This function does not support nested VMs.
+ *
+ * HCR has lots of settings, some of which differ between vCPUs, but most of
+ * them control traps. For EL1 VMs, the bits relevant to MMU translation are
+ * common to all vCPUs:
+ * - VM, PTW, DC, TGE, CD, ID, NV, NV1, DCT have fixed values.
+ * - E2H, FWB, ATA depend on host CPU features (respectively VHE, FWB and MTE2).
+ * - RW depends on vCPU features. This function does not support 32-bit VMs.
+ *
+ * HCRX: EnSDERR, EnSNERR set to 0. TODO: is GCSEn relevant?
+ *
+ * VTCR is set once for the VM, depends on the VM's IPA size.
+ *
+ * VTTBR:
+ * - VMID is pinned, until kvm_arch_put_guest_mmu() is called.
+ * - PGD is allocated and freed with the vm (the caller must hold their
+ * reference to @kvm). KVM may create and remove page table mappings at any
+ * moment, and uses normal TLB maintenance operations to synchronize with
+ * page table walkers. KVM does not notify the caller about mapping changes.
+ * - CNP is a property of the host system.
+ *
+ * Return: a new kvm_arch_guest_mmu instance, or an error pointer on failure.
+ */
+struct kvm_arch_guest_mmu *kvm_arch_get_guest_mmu(struct kvm *kvm)
+{
+ int ret;
+ struct kvm_arch_guest_mmu *mmu;
+
+ guard(mutex)(&kvm->arch.config_lock);
+
+ if (is_protected_kvm_enabled())
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (!test_bit(KVM_ARCH_FLAG_VCPU_FEATURES_CONFIGURED, &kvm->arch.flags) ||
+ test_bit(KVM_ARM_VCPU_EL1_32BIT, kvm->arch.vcpu_features) ||
+ test_bit(KVM_ARM_VCPU_HAS_EL2, kvm->arch.vcpu_features))
+ return ERR_PTR(-EINVAL);
+
+ ret = kvm_arm_pinned_vmid_get(kvm);
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ ret = -ENOMEM;
+ mmu = kzalloc_obj(*mmu, GFP_KERNEL_ACCOUNT);
+ if (!mmu)
+ goto err_unpin_vmid;
+
+ mmu->vtcr = kvm->arch.mmu.vtcr;
+ mmu->vttbr = kvm_get_vttbr(&kvm->arch.mmu);
+ mmu->hcr = kvm_mmu_get_hcr(kvm, 0);
+
+ return mmu;
+
+err_unpin_vmid:
+ kvm_arm_pinned_vmid_put(kvm);
+ return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(kvm_arch_get_guest_mmu);
+
+void kvm_arch_put_guest_mmu(struct kvm *kvm, struct kvm_arch_guest_mmu *mmu)
+{
+ kvm_arm_pinned_vmid_put(kvm);
+ kfree(mmu);
+}
+EXPORT_SYMBOL_GPL(kvm_arch_put_guest_mmu);
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index a2f4e769a428..20277f888380 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -5798,8 +5798,8 @@ static void vcpu_set_hcr(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = vcpu->kvm;
- if (has_vhe() || has_hvhe())
- vcpu->arch.hcr_el2 |= HCR_E2H;
+ vcpu->arch.hcr_el2 = kvm_mmu_get_hcr(kvm, vcpu->arch.hcr_el2);
+
if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN)) {
/* route synchronous external abort exceptions to EL2 */
vcpu->arch.hcr_el2 |= HCR_TEA;
@@ -5807,9 +5807,6 @@ static void vcpu_set_hcr(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 |= HCR_TERR;
}
- if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
- vcpu->arch.hcr_el2 |= HCR_FWB;
-
if (cpus_have_final_cap(ARM64_HAS_EVT) &&
!cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE) &&
kvm_read_vm_id_reg(kvm, SYS_CTR_EL0) == read_sanitised_ftr_reg(SYS_CTR_EL0))
@@ -5817,12 +5814,7 @@ static void vcpu_set_hcr(struct kvm_vcpu *vcpu)
else
vcpu->arch.hcr_el2 |= HCR_TID2;
- if (vcpu_el1_is_32bit(vcpu))
- vcpu->arch.hcr_el2 &= ~HCR_RW;
-
- if (kvm_has_mte(vcpu->kvm))
- vcpu->arch.hcr_el2 |= HCR_ATA;
- else
+ if (!kvm_has_mte(vcpu->kvm))
vcpu->arch.hcr_el2 |= HCR_TID5;
/*
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 12/15] vfio: Support VFIO devices that access the CPU MMU
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (10 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 11/15] KVM: arm64: Export stage-2 MMU config Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 16:44 ` Jason Gunthorpe
2026-10-15 14:25 ` [RFC PATCH v2 13/15] misc/arm-cla: Add stage-2 MTC and VM context switch Ryan Roberts
` (2 subsequent siblings)
14 siblings, 1 reply; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Support devices that are attached to a CPU and perform DMA through a
MMU. They do not have an IOMMU and do not use a VFIO container.
The VMM needs device-specific knowledge in order to create a virtual
device (device tree, ACPI tables), so it also knows not to allocate a
container.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/vfio/group.c | 37 +++++++++++++++++++++++++++++--------
drivers/vfio/vfio.h | 5 +++++
drivers/vfio/vfio_main.c | 16 +++++++++++++++-
include/linux/vfio.h | 1 +
4 files changed, 50 insertions(+), 9 deletions(-)
diff --git a/drivers/vfio/group.c b/drivers/vfio/group.c
index b2299e5bc6df..99d85880a41c 100644
--- a/drivers/vfio/group.c
+++ b/drivers/vfio/group.c
@@ -120,7 +120,7 @@ static int vfio_group_ioctl_set_container(struct vfio_group *group,
return -EBADF;
mutex_lock(&group->group_lock);
- if (vfio_group_has_iommu(group)) {
+ if (vfio_group_has_iommu(group) || group->type == VFIO_CPU_MMU) {
ret = -EINVAL;
goto out_unlock;
}
@@ -173,7 +173,8 @@ static int vfio_df_group_open(struct vfio_device_file *df)
int ret;
mutex_lock(&device->group->group_lock);
- if (!vfio_group_has_iommu(device->group)) {
+ if (!vfio_group_has_iommu(device->group) &&
+ device->group->type != VFIO_CPU_MMU) {
ret = -EINVAL;
goto out_unlock;
}
@@ -635,7 +636,8 @@ static bool vfio_group_has_device(struct vfio_group *group, struct device *dev)
return false;
}
-static struct vfio_group *vfio_group_find_or_alloc(struct device *dev)
+static struct vfio_group *vfio_group_find_or_alloc(struct device *dev,
+ enum vfio_group_type type)
{
struct iommu_group *iommu_group;
struct vfio_group *group;
@@ -662,12 +664,13 @@ static struct vfio_group *vfio_group_find_or_alloc(struct device *dev)
mutex_lock(&vfio.group_lock);
group = vfio_group_find_from_iommu(iommu_group);
if (group) {
- if (WARN_ON(vfio_group_has_device(group, dev)))
+ if (WARN_ON(vfio_group_has_device(group, dev)) ||
+ WARN_ON(group->type != type))
group = ERR_PTR(-EINVAL);
else
refcount_inc(&group->drivers);
} else {
- group = vfio_create_group(iommu_group, VFIO_IOMMU);
+ group = vfio_create_group(iommu_group, type);
}
mutex_unlock(&vfio.group_lock);
@@ -681,10 +684,18 @@ int vfio_device_set_group(struct vfio_device *device,
{
struct vfio_group *group;
- if (type == VFIO_IOMMU)
- group = vfio_group_find_or_alloc(device->dev);
- else
+ switch (type) {
+ case VFIO_IOMMU:
+ case VFIO_CPU_MMU:
+ group = vfio_group_find_or_alloc(device->dev, type);
+ break;
+ case VFIO_EMULATED_IOMMU:
+ case VFIO_NO_IOMMU:
group = vfio_noiommu_group_alloc(device->dev, type);
+ break;
+ default:
+ group = ERR_PTR(-EINVAL);
+ }
if (IS_ERR(group))
return PTR_ERR(group);
@@ -761,6 +772,11 @@ int vfio_device_group_use_iommu(struct vfio_device *device)
lockdep_assert_held(&group->group_lock);
+ if (group->type == VFIO_CPU_MMU) {
+ get_file(group->opened_file);
+ return 0;
+ }
+
if (WARN_ON(!group->container))
return -EINVAL;
@@ -777,6 +793,11 @@ void vfio_device_group_unuse_iommu(struct vfio_device *device)
lockdep_assert_held(&group->group_lock);
+ if (group->type == VFIO_CPU_MMU) {
+ fput(group->opened_file);
+ return;
+ }
+
if (WARN_ON(!group->container))
return;
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index 7728bc99b63d..07c5223612d3 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -64,6 +64,11 @@ enum vfio_group_type {
* taint the kernel.
*/
VFIO_NO_IOMMU,
+
+ /*
+ * Physical devices using the CPU MMU
+ */
+ VFIO_CPU_MMU,
};
#if IS_ENABLED(CONFIG_VFIO_GROUP)
diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 423ead48aafe..41fee81a14cb 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -344,7 +344,8 @@ static int __vfio_register_dev(struct vfio_device *device,
{
int ret;
- if (WARN_ON(IS_ENABLED(CONFIG_IOMMUFD) &&
+ if (type != VFIO_CPU_MMU &&
+ WARN_ON(IS_ENABLED(CONFIG_IOMMUFD) &&
(!device->ops->bind_iommufd ||
!device->ops->unbind_iommufd ||
!device->ops->attach_ioas ||
@@ -415,6 +416,19 @@ int vfio_register_emulated_iommu_dev(struct vfio_device *device)
}
EXPORT_SYMBOL_GPL(vfio_register_emulated_iommu_dev);
+/*
+ * Register a devices that accesses memory through the CPU MMU. Although it
+ * doesn't use an IOMMU, the device must have an IOMMU group.
+ */
+int vfio_register_cpu_mmu_dev(struct vfio_device *device)
+{
+ if (!IS_ENABLED(CONFIG_VFIO_GROUP))
+ return -EOPNOTSUPP;
+
+ return __vfio_register_dev(device, VFIO_CPU_MMU);
+}
+EXPORT_SYMBOL_GPL(vfio_register_cpu_mmu_dev);
+
/*
* Decrement the device reference count and wait for the device to be
* removed. Open file descriptors for the device... */
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 45f08986359e..7d3a5ff1ad00 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -337,6 +337,7 @@ static inline void vfio_put_device(struct vfio_device *device)
put_device(&device->device);
}
+int vfio_register_cpu_mmu_dev(struct vfio_device *device);
int vfio_register_group_dev(struct vfio_device *device);
int vfio_register_emulated_iommu_dev(struct vfio_device *device);
void vfio_unregister_group_dev(struct vfio_device *device);
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC PATCH v2 12/15] vfio: Support VFIO devices that access the CPU MMU
2026-10-15 14:25 ` [RFC PATCH v2 12/15] vfio: Support VFIO devices that access the CPU MMU Ryan Roberts
@ 2026-10-15 16:44 ` Jason Gunthorpe
0 siblings, 0 replies; 20+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 16:44 UTC (permalink / raw)
To: Ryan Roberts
Cc: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet, linux-kernel,
linux-arm-kernel, kvmarm, kvm, devicetree, dri-devel, linux-doc
On Thu, Oct 15, 2026 at 03:25:46PM +0100, Ryan Roberts wrote:
> From: Jean-Philippe Brucker <jpb@kernel.org>
>
> Support devices that are attached to a CPU and perform DMA through a
> MMU. They do not have an IOMMU and do not use a VFIO container.
> The VMM needs device-specific knowledge in order to create a virtual
> device (device tree, ACPI tables), so it also knows not to allocate a
> container.
I'm not so excited to see new things built on top of the group
interface.
Maybe in this case you just don't have to do a bind operation on the
cdev to an iommufd? That feels kind of natural if you don't use a
iommufd
However.. you still have to affirm which single mm is the mm that the
device is going to be doing DMA against. The process the opens the fd
should not be forced to be the mm that it gets linked to.
From that perspective maybe you should create an iommufd and a special
"SVA" IOAS that it can attach to. This is more complicated but it
makes the mm linkage and blocking beahvior explicit and controlled by
userspace.
Jason
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC PATCH v2 13/15] misc/arm-cla: Add stage-2 MTC and VM context switch
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (11 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 12/15] vfio: Support VFIO devices that access the CPU MMU Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 14/15] misc/arm-cla: Support exclusive context assignment Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 15/15] misc/arm-cla: Add vfio-mdev user interface Ryan Roberts
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Add functions to switch the guest MTC. Reset the accelerators and CLA
registers on switch. No save/restore of accelerators at the moment, we
only restore the kernel's PL0CTL.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/arm-cla-regs.h | 12 ++
drivers/misc/arm-cla/arm-cla.h | 41 ++++++-
drivers/misc/arm-cla/cla-ctx.c | 55 +++++++++-
drivers/misc/arm-cla/cla-mtc.c | 163 +++++++++++++++++++++++++---
drivers/misc/arm-cla/cla-regs.c | 63 ++++++++++-
drivers/misc/arm-cla/cla-sched.c | 5 +-
6 files changed, 308 insertions(+), 31 deletions(-)
diff --git a/drivers/misc/arm-cla/arm-cla-regs.h b/drivers/misc/arm-cla/arm-cla-regs.h
index 167e9ab64051..a7fc19f67893 100644
--- a/drivers/misc/arm-cla/arm-cla-regs.h
+++ b/drivers/misc/arm-cla/arm-cla-regs.h
@@ -37,6 +37,18 @@
#define CLA_PLxCTRL_DBGPERM GENMASK(3, 1)
#define CLA_PLxCTRL_PREP(accid, v) ((u64)(v) << (8 * (accid)))
+/* Valid bits of PLxCTRL. Other bits are RAZ/WI */
+#define _CLA_PLxCTRL_MASK (CLA_PLxCTRL_AVAIL | CLA_PLxCTRL_DBGPERM)
+#define CLA_PLxCTRL_MASK \
+ (CLA_PLxCTRL_PREP(0, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(1, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(2, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(3, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(4, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(5, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(6, _CLA_PLxCTRL_MASK) | \
+ CLA_PLxCTRL_PREP(7, _CLA_PLxCTRL_MASK))
+
#define CLA_STATUS_AVAIL BIT(0)
#define CLA_STATUS_DMB BIT(1)
#define CLA_STATUS_EABORT BIT(2)
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index e047094dd49c..c9ecab5ab6bf 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -31,6 +31,9 @@
/* Quantum of CLA assignment */
#define CLA_SLICE_MS 100
+struct kvm;
+struct kvm_arch_guest_mmu;
+
struct cla_domain;
struct cla_call_on_cpu {
@@ -141,18 +144,43 @@ struct cla_regs_pl0 {
u64 regstate[];
};
+/**
+ * struct cla_regs_pl1 - Saved CLA register state (PL1)
+ * @pl0ctrl: Host PL0 control register
+ *
+ * Most registers are simply reset when transitionning to and from VMs, since we
+ * don't support shared assignment.
+ */
+struct cla_regs_pl1 {
+ u64 pl0ctrl;
+};
+
+enum cla_ctx_type {
+ CLA_CTX_PL0,
+ CLA_CTX_PL1,
+};
+
/**
* struct cla_ctx_key - Unique context identifier
+ * @type: Context type, PL0 or PL1
* @mm: For PL0 contexts, the mm.
* @file: for PL0 contexts, the file.
+ * @kvm: For PL1 contexts, the virtual machine.
*/
struct cla_ctx_key {
- struct mm_struct *mm;
- struct file *file;
+ enum cla_ctx_type type;
+ union {
+ struct {
+ struct mm_struct *mm;
+ struct file *file;
+ };
+ struct kvm *kvm;
+ };
};
#define CLA_CTX_KEY_PL0(_mm, _file) \
{ \
+ .type = CLA_CTX_PL0, \
.mm = (_mm), \
.file = (_file), \
}
@@ -180,6 +208,7 @@ struct cla_ctx_key {
* Mutable, written by domain reassignment, device work, and CPU hotplug:
* @regs: State for each device in domain.
* @asid: Pinned ASID of live context.
+ * @guest_mmu: Stage 2 translation configuration.
*/
struct cla_ctx {
struct kref refcnt;
@@ -191,7 +220,10 @@ struct cla_ctx {
refcount_t users;
struct list_head queue_node;
void **regs;
- unsigned int asid;
+ union {
+ unsigned int asid;
+ struct kvm_arch_guest_mmu *guest_mmu;
+ };
wait_queue_head_t waitq;
};
@@ -268,6 +300,9 @@ static inline bool cla_ctx_is_dying(struct cla_ctx *ctx)
return refcount_read(&ctx->users) == 0;
}
+#define cla_ctx_is_pl0(ctx) ((ctx)->key.type == CLA_CTX_PL0)
+#define cla_ctx_is_pl1(ctx) ((ctx)->key.type == CLA_CTX_PL1)
+
void cla_dev_switch_ctx_work(struct kthread_work *work);
int cla_dev_switch_ctx(struct cla_dev *dev, struct cla_ctx *prev_ctx,
struct cla_ctx *next_ctx);
diff --git a/drivers/misc/arm-cla/cla-ctx.c b/drivers/misc/arm-cla/cla-ctx.c
index 6a9c95769724..0489bf7a103f 100644
--- a/drivers/misc/arm-cla/cla-ctx.c
+++ b/drivers/misc/arm-cla/cla-ctx.c
@@ -3,11 +3,42 @@
#include <linux/err.h>
#include <linux/file.h>
#include <linux/gfp.h>
+#include <linux/kvm_host.h>
#include <linux/sched/mm.h>
#include <linux/slab.h>
#include "arm-cla.h"
+#if IS_ENABLED(CONFIG_KVM)
+/*
+ * We must hold a reference to KVM at least as long as the PL1 context is
+ * assigned. We hold it until the ctx is freed.
+ */
+static inline bool cla_ctx_get_pl1(struct cla_ctx *ctx)
+{
+ return kvm_get_kvm_safe((ctx)->key.kvm);
+}
+
+static inline void cla_ctx_put_pl1(struct cla_ctx *ctx)
+{
+ kvm_put_kvm((ctx)->key.kvm);
+}
+#else
+/*
+ * If KVM isn't enabled, the driver doesn't provide a user interface to create a
+ * PL1 context, so these are never called.
+ */
+static inline bool cla_ctx_get_pl1(struct cla_ctx *ctx)
+{
+ return false;
+}
+
+static inline void cla_ctx_put_pl1(struct cla_ctx *ctx)
+{
+ WARN_ON(1);
+}
+#endif
+
/**
* cla_ctx_create - Create a context or increase its user count
* @domain: CLA domain
@@ -48,8 +79,12 @@ struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
INIT_LIST_HEAD(&ctx->queue_node);
init_waitqueue_head(&ctx->waitq);
- mmgrab(key->mm);
- get_file(key->file);
+ if (cla_ctx_is_pl0(ctx)) {
+ mmgrab(key->mm);
+ get_file(key->file);
+ } else {
+ WARN_ON(!cla_ctx_get_pl1(ctx));
+ }
/*
* Test if the mm or kvm is compatible. If cla_mtc_get_ctx() fails
@@ -74,8 +109,12 @@ struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
err_free_regs:
cla_ctx_free_regs(domain, ctx);
err_free:
- mmdrop(key->mm);
- fput(key->file);
+ if (cla_ctx_is_pl0(ctx)) {
+ mmdrop(key->mm);
+ fput(key->file);
+ } else {
+ cla_ctx_put_pl1(ctx);
+ }
kfree(ctx);
err_unlock:
mutex_unlock(&domain->lock);
@@ -139,8 +178,12 @@ void cla_ctx_free(struct kref *ref)
struct cla_ctx *ctx = container_of(ref, struct cla_ctx, refcnt);
WARN_ON(!cla_ctx_is_dying(ctx));
- mmdrop(ctx->key.mm);
- fput(ctx->key.file);
+ if (cla_ctx_is_pl0(ctx)) {
+ mmdrop(ctx->key.mm);
+ fput(ctx->key.file);
+ } else {
+ cla_ctx_put_pl1(ctx);
+ }
cla_ctx_free_regs(ctx->domain, ctx);
kfree(ctx);
}
diff --git a/drivers/misc/arm-cla/cla-mtc.c b/drivers/misc/arm-cla/cla-mtc.c
index 9ab25085dfdf..fc2cacb09c4a 100644
--- a/drivers/misc/arm-cla/cla-mtc.c
+++ b/drivers/misc/arm-cla/cla-mtc.c
@@ -7,12 +7,64 @@
#include <asm/kvm_arm.h>
#include <linux/mmu_context.h>
+#include <linux/kvm_host.h>
#include "arm-cla.h"
#define cla_kernel_mtc \
(cla_kernel_pl == 2 ? CLA_MTC_REGIDX_PL2 : CLA_MTC_REGIDX_PL1)
+/**
+ * struct cla_mtc_s2 - Stage 2 MTC configuration for the host or a VM.
+ * @hcr: HCR_EL2 register value.
+ * @vttbr: VTTBR_EL2 register value.
+ * @vtcr: VTCR_EL2 register value.
+ */
+struct cla_mtc_s2 {
+ u64 hcr;
+ u64 vttbr;
+ u64 vtcr;
+};
+
+#if IS_ENABLED(CONFIG_KVM)
+static void cla_mtc_prep_s2(struct cla_mtc_s2 *s2, struct cla_ctx *ctx)
+{
+ /* See comment in cla_mtc_setup() about MTC register definitions. */
+ s2->hcr = ctx->guest_mmu->hcr;
+ s2->vtcr = ctx->guest_mmu->vtcr;
+ s2->vttbr = ctx->guest_mmu->vttbr;
+}
+
+static int cla_mtc_get_s2_ctx(struct cla_ctx *ctx)
+{
+ struct kvm_arch_guest_mmu *mmu = kvm_arch_get_guest_mmu(ctx->key.kvm);
+
+ if (IS_ERR(mmu))
+ return PTR_ERR(mmu);
+ ctx->guest_mmu = mmu;
+
+ return 0;
+}
+
+static void cla_mtc_put_s2_ctx(struct cla_ctx *ctx)
+{
+ kvm_arch_put_guest_mmu(ctx->key.kvm, ctx->guest_mmu);
+}
+
+static void cla_mtc_leak_s2_ctx(struct cla_ctx *ctx)
+{
+ kvm_get_kvm(ctx->key.kvm);
+}
+#else
+static void cla_mtc_prep_s2(struct cla_mtc_s2 *s2, struct cla_ctx *ctx) {}
+static int cla_mtc_get_s2_ctx(struct cla_ctx *ctx)
+{
+ return -EOPNOTSUPP;
+}
+static void cla_mtc_put_s2_ctx(struct cla_ctx *ctx) {}
+static void cla_mtc_leak_s2_ctx(struct cla_ctx *ctx) {}
+#endif
+
/**
* cla_mtc_setup - initialize the memory translation context
* @dev: CLA device
@@ -85,22 +137,21 @@ int cla_mtc_setup(struct cla_dev *dev)
return 0;
}
-/**
- * cla_mtc_clear - clear the memory translation context
- * @dev: CLA device
- *
- * Return: 0 on success, or a negative error code.
- */
-int cla_mtc_clear(struct cla_dev *dev)
+static int __cla_mtc_clear(struct cla_dev *dev, bool include_s2)
{
/* Don't set the RES1 bit in VTCR_EL2 because in MTC they are RAZ/WI. */
u64 regs[2 * CLA_MTC_PL_SIZE] = {};
- size_t regs_size = cla_kernel_pl * CLA_MTC_PL_SIZE;
+ size_t regs_size;
/* SETCTX fails if there is no ACCID */
if (!dev->accelerators)
return 0;
+ if (include_s2)
+ regs_size = cla_kernel_pl * CLA_MTC_PL_SIZE;
+ else
+ regs_size = CLA_MTC_PL_SIZE;
+
/*
* All accelerators are idle, meaning there can't be any memory
* transactions happening. So it is safe to clear the MTC in any order.
@@ -109,6 +160,23 @@ int cla_mtc_clear(struct cla_dev *dev)
return cla_op_setctx(dev, CLA_MTC_REGIDX_PL1, regs_size, regs);
}
+/**
+ * cla_mtc_clear - clear the whole memory translation context
+ * @dev: CLA device
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_clear(struct cla_dev *dev)
+{
+ return __cla_mtc_clear(dev, true);
+}
+
+/* Clear the EL1&0 stage 1 MTC config */
+static int cla_mtc_clear_s1(struct cla_dev *dev)
+{
+ return __cla_mtc_clear(dev, false);
+}
+
static int cla_mtc_install_s1(struct cla_dev *dev, struct cla_ctx *ctx)
{
u64 ttbr0;
@@ -137,6 +205,49 @@ static int cla_mtc_uninstall_s1(struct cla_dev *dev)
return cla_op_setctx(dev, cla_kernel_mtc + CLA_MTC_TTBR0, 1, &ttbr0);
}
+static int cla_mtc_install_s2(struct cla_dev *dev, struct cla_ctx *ctx)
+{
+ struct cla_mtc_s2 s2 = {};
+ const size_t num_regs = sizeof(s2) / sizeof(u64);
+
+ if (WARN_ON(cla_kernel_pl != 2))
+ return -EINVAL;
+
+ /* GETCTX/SETCTX fail if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ cla_mtc_prep_s2(&s2, ctx);
+
+ return cla_op_setctx(dev, CLA_MTC_HCR_EL2, num_regs, (u64 *)&s2);
+}
+
+static int cla_mtc_uninstall_s2(struct cla_dev *dev)
+{
+ struct cla_mtc_s2 s2 = {
+ .hcr = HCR_HOST_VHE_FLAGS,
+ };
+ const size_t num_regs = sizeof(s2) / sizeof(u64);
+ int ret;
+
+ if (WARN_ON(cla_kernel_pl != 2))
+ return -EINVAL;
+
+ /* GETCTX/SETCTX fail if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ /*
+ * The accelerators are idle and there is no pending memory transaction,
+ * so we can change the MTC config in any order.
+ */
+ ret = cla_mtc_clear_s1(dev);
+ if (ret)
+ return ret;
+
+ return cla_op_setctx(dev, CLA_MTC_HCR_EL2, num_regs, (u64 *)&s2);
+}
+
/**
* cla_mtc_install - enable a translation context
* @dev: CLA device
@@ -146,7 +257,8 @@ static int cla_mtc_uninstall_s1(struct cla_dev *dev)
*/
int cla_mtc_install(struct cla_dev *dev, struct cla_ctx *ctx)
{
- return cla_mtc_install_s1(dev, ctx);
+ return cla_ctx_is_pl0(ctx) ?
+ cla_mtc_install_s1(dev, ctx) : cla_mtc_install_s2(dev, ctx);
}
/**
@@ -158,7 +270,8 @@ int cla_mtc_install(struct cla_dev *dev, struct cla_ctx *ctx)
*/
int cla_mtc_uninstall(struct cla_dev *dev, struct cla_ctx *ctx)
{
- return cla_mtc_uninstall_s1(dev);
+ return cla_ctx_is_pl0(ctx) ?
+ cla_mtc_uninstall_s1(dev) : cla_mtc_uninstall_s2(dev);
}
/**
@@ -166,15 +279,23 @@ int cla_mtc_uninstall(struct cla_dev *dev, struct cla_ctx *ctx)
* @ctx: context to get
*
* Grab the information needed for installing this context.
- * For stage 1 MTC, pin the ASID.
+ * For stage 1 MTC, pin the ASID. For stage 2 MTC, pin the VMID and get the
+ * translation configuration registers.
*
* Return: 0 on success, or a negative error code.
*/
int cla_mtc_get_ctx(struct cla_ctx *ctx)
{
- ctx->asid = arm64_mm_context_get(ctx->key.mm);
- if (ctx->asid == 0)
- return -ENOSPC;
+ if (cla_ctx_is_pl0(ctx)) {
+ ctx->asid = arm64_mm_context_get(ctx->key.mm);
+ if (ctx->asid == 0)
+ return -ENOSPC;
+ } else {
+ int ret = cla_mtc_get_s2_ctx(ctx);
+
+ if (ret)
+ return ret;
+ }
return 0;
}
@@ -187,7 +308,10 @@ int cla_mtc_get_ctx(struct cla_ctx *ctx)
*/
void cla_mtc_put_ctx(struct cla_ctx *ctx)
{
- arm64_mm_context_put(ctx->key.mm);
+ if (cla_ctx_is_pl0(ctx))
+ arm64_mm_context_put(ctx->key.mm);
+ else
+ cla_mtc_put_s2_ctx(ctx);
}
/**
@@ -201,6 +325,11 @@ void cla_mtc_put_ctx(struct cla_ctx *ctx)
*/
void cla_mtc_leak_ctx(struct cla_ctx *ctx)
{
- WARN(1, "leaking PL0 context\n");
- mmgrab(ctx->key.mm);
+ if (cla_ctx_is_pl0(ctx)) {
+ WARN(1, "leaking PL0 context\n");
+ mmgrab(ctx->key.mm);
+ } else {
+ WARN(1, "leaking PL1 context\n");
+ cla_mtc_leak_s2_ctx(ctx);
+ }
}
diff --git a/drivers/misc/arm-cla/cla-regs.c b/drivers/misc/arm-cla/cla-regs.c
index d48311d0ef92..5be5d1d4f4e4 100644
--- a/drivers/misc/arm-cla/cla-regs.c
+++ b/drivers/misc/arm-cla/cla-regs.c
@@ -205,6 +205,55 @@ static int cla_regs_install_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
return 0;
}
+static int cla_regs_uninstall_pl1(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id)
+{
+ int ret;
+ u64 lresp;
+ struct cla_regs_pl1 *regs = ctx->regs[ctx_id];
+
+ /* VM may have been killed while it was launching ops */
+ ret = cla_op_wait_lresp(dev, &lresp);
+ if (ret)
+ return ret;
+
+ /*
+ * TODO: PL1 context save/restore is not supported. Reset the VM state
+ * even when the context remains live across CPU hotplug or a VFIO
+ * reset.
+ */
+ ret = cla_op_reset_all(dev);
+ if (ret)
+ return ret;
+
+ /* Restore kernel PL0CTRL */
+ cla_reg_write(dev, CLA_REG_PL0CTRL, regs->pl0ctrl);
+
+ return 0;
+}
+
+static int cla_regs_install_pl1(struct cla_dev *dev, struct cla_ctx *ctx,
+ unsigned int ctx_id)
+{
+ int i;
+ unsigned int accid;
+ struct cla_regs_pl1 *regs = ctx->regs[ctx_id];
+
+ /* Save PL0CTRL and initialize it as all-enabled */
+ regs->pl0ctrl = cla_reg_read(dev, CLA_REG_PL0CTRL);
+ cla_reg_write(dev, CLA_REG_PL0CTRL, CLA_PLxCTRL_MASK);
+
+ /* Clear EVENT by reading STATUS, to provide a pristine context */
+ cla_for_each_accid(dev, accid)
+ cla_reg_read(dev, CLA_REG_STATUS(accid));
+
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), 0);
+ cla_reg_write(dev, CLA_REG_LRESP, 0);
+
+ return 0;
+}
+
/**
* cla_regs_install - Install and/or restore CLA context
* @dev: CLA device
@@ -215,6 +264,7 @@ static int cla_regs_install_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
*
* PL0 context: Restore the DATA and LRESP registers, and accelerator state if
* one has been saved.
+ * PL1 context: Save host kernel registers. Reset all PL1 and PL0 state.
*
* This function is called with all accelerators idle and no trace of previous
* work.
@@ -224,7 +274,9 @@ static int cla_regs_install_pl0(struct cla_dev *dev, struct cla_ctx *ctx,
int cla_regs_install(struct cla_dev *dev, struct cla_ctx *ctx,
unsigned int ctx_id)
{
- return cla_regs_install_pl0(dev, ctx, ctx_id);
+ return cla_ctx_is_pl0(ctx) ?
+ cla_regs_install_pl0(dev, ctx, ctx_id) :
+ cla_regs_install_pl1(dev, ctx, ctx_id);
}
/**
@@ -237,6 +289,7 @@ int cla_regs_install(struct cla_dev *dev, struct cla_ctx *ctx,
*
* PL0 context: Save the DATA registers and accelerator state unless
* cla_ctx_is_dying() is true, since a dying context will not be restored.
+ * PL1 context: Restore host kernel registers.
*
* When this completes, all accelerators are idle. If this fails, some
* accelerators may still be running.
@@ -246,7 +299,9 @@ int cla_regs_install(struct cla_dev *dev, struct cla_ctx *ctx,
int cla_regs_uninstall(struct cla_dev *dev, struct cla_ctx *ctx,
unsigned int ctx_id)
{
- return cla_regs_uninstall_pl0(dev, ctx, ctx_id);
+ return cla_ctx_is_pl0(ctx) ?
+ cla_regs_uninstall_pl0(dev, ctx, ctx_id) :
+ cla_regs_uninstall_pl1(dev, ctx, ctx_id);
}
/**
@@ -279,7 +334,9 @@ int cla_ctx_alloc_regs(struct cla_domain *domain, struct cla_ctx *ctx)
* The regs structures are only ever accessed from the CLA
* device's CPU, so try to allocate them on the right NUMA node
*/
- size = struct_size_t(struct cla_regs_pl0, regstate, nr_regstate);
+ size = cla_ctx_is_pl0(ctx) ?
+ struct_size_t(struct cla_regs_pl0, regstate, nr_regstate) :
+ sizeof(struct cla_regs_pl1);
regs = kvzalloc_node(size, GFP_KERNEL_ACCOUNT,
cpu_to_node(dev->cpu));
if (!regs)
diff --git a/drivers/misc/arm-cla/cla-sched.c b/drivers/misc/arm-cla/cla-sched.c
index 6b5b782ac33e..dc0584d681af 100644
--- a/drivers/misc/arm-cla/cla-sched.c
+++ b/drivers/misc/arm-cla/cla-sched.c
@@ -196,9 +196,10 @@ static void cla_domain_zap(struct cla_domain *domain, struct cla_ctx *ctx)
/*
* If the context is dying it has already been unmapped, no need to zap
- * it here.
+ * it here. PL1 contexts are not bound to a specific mm, but since
+ * deassigning them requires unmapping them first, they must be dying.
*/
- if (!ctx || cla_ctx_is_dying(ctx))
+ if (!ctx || cla_ctx_is_dying(ctx) || WARN_ON(!cla_ctx_is_pl0(ctx)))
return;
if (!mmget_not_zero(ctx->key.mm))
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 14/15] misc/arm-cla: Support exclusive context assignment
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (12 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 13/15] misc/arm-cla: Add stage-2 MTC and VM context switch Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 14:25 ` [RFC PATCH v2 15/15] misc/arm-cla: Add vfio-mdev user interface Ryan Roberts
14 siblings, 0 replies; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Introduce a nr_contexts members to cla_domain to count the mapped
contexts. At the moment PL0 contexts share the domain, while PL1
contexts require exclusive assignment, since there is no context
save/restore. A negative count indicates the domain is exclusive.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/arm-cla.h | 2 ++
drivers/misc/arm-cla/cla-ctx.c | 44 ++++++++++++++++++++++++++++++++--
2 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index c9ecab5ab6bf..43a58a329525 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -105,6 +105,7 @@ struct cla_dev {
* @lock: Protects the following members.
* @ctxs: All live contexts, keyed on cla_ctx_key.
* @queued_ctxs: Queue of contexts waiting for assignment.
+ * @nr_ctxs: Context count. Negative value for exclusive assignment.
* @worker: Kthread worker to coordinate reassignment.
* @reassign: Delayed work that switches contexts with time slicing.
*
@@ -120,6 +121,7 @@ struct cla_domain {
struct mutex lock;
bool broken;
+ long nr_ctxs;
struct rhashtable ctxs;
struct list_head queued_ctxs;
struct cla_ctx *assigned_ctx;
diff --git a/drivers/misc/arm-cla/cla-ctx.c b/drivers/misc/arm-cla/cla-ctx.c
index 0489bf7a103f..f956cc88909f 100644
--- a/drivers/misc/arm-cla/cla-ctx.c
+++ b/drivers/misc/arm-cla/cla-ctx.c
@@ -39,6 +39,38 @@ static inline void cla_ctx_put_pl1(struct cla_ctx *ctx)
}
#endif
+static int cla_domain_get(struct cla_domain *domain, bool share)
+{
+ lockdep_assert_held(&domain->lock);
+
+ if (share) {
+ /* Time share the domain with other contexts. */
+ if (domain->nr_ctxs < 0)
+ return -EBUSY;
+ domain->nr_ctxs++;
+ WARN_ON(domain->nr_ctxs < 0); /* LONG_MAX contexts?! */
+ } else {
+ /* Reserve the domain for exclusive assignment. */
+ if (domain->nr_ctxs != 0)
+ return -EBUSY;
+ domain->nr_ctxs--;
+ }
+ return 0;
+}
+
+/* Undo cla_domain_get() */
+static void cla_domain_put(struct cla_domain *domain)
+{
+ lockdep_assert_held(&domain->lock);
+
+ if (domain->nr_ctxs == -1)
+ domain->nr_ctxs = 0;
+ else if (domain->nr_ctxs > 0)
+ domain->nr_ctxs--;
+ else
+ WARN(1, "invalid domain refcount %ld\n", domain->nr_ctxs);
+}
+
/**
* cla_ctx_create - Create a context or increase its user count
* @domain: CLA domain
@@ -53,7 +85,7 @@ struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
struct cla_ctx_key *key)
{
struct cla_ctx *ctx;
- int ret = -ENOMEM;
+ int ret;
mutex_lock(&domain->lock);
ctx = cla_domain_lookup_ctx(domain, key);
@@ -68,9 +100,14 @@ struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
goto err_unlock;
}
+ ret = cla_domain_get(domain, key->type == CLA_CTX_PL0);
+ if (ret)
+ goto err_unlock;
+
+ ret = -ENOMEM;
ctx = kzalloc_obj(*ctx, GFP_KERNEL_ACCOUNT);
if (!ctx)
- goto err_unlock;
+ goto err_put_domain;
kref_init(&ctx->refcnt);
refcount_set(&ctx->users, 1);
@@ -116,6 +153,8 @@ struct cla_ctx *cla_ctx_create(struct cla_domain *domain,
cla_ctx_put_pl1(ctx);
}
kfree(ctx);
+err_put_domain:
+ cla_domain_put(domain);
err_unlock:
mutex_unlock(&domain->lock);
return ERR_PTR(ret);
@@ -161,6 +200,7 @@ void cla_ctx_destroy(struct cla_domain *domain, struct cla_ctx_key *key)
*/
if (domain->assigned_ctx == ctx)
cla_domain_schedule_reassignment(domain);
+ cla_domain_put(domain);
mutex_unlock(&domain->lock);
/* Release initial reference from cla_ctx_create(). */
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* [RFC PATCH v2 15/15] misc/arm-cla: Add vfio-mdev user interface
2026-10-15 14:25 [RFC PATCH v2 00/15] Arm Core Local Accelerator Driver Ryan Roberts
` (13 preceding siblings ...)
2026-10-15 14:25 ` [RFC PATCH v2 14/15] misc/arm-cla: Support exclusive context assignment Ryan Roberts
@ 2026-10-15 14:25 ` Ryan Roberts
2026-10-15 16:41 ` Jason Gunthorpe
14 siblings, 1 reply; 20+ messages in thread
From: Ryan Roberts @ 2026-10-15 14:25 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Jason Gunthorpe, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, kvmarm, kvm,
devicetree, dri-devel, linux-doc
From: Jean-Philippe Brucker <jpb@kernel.org>
Implement an interface to assign CLA devices to KVM guests. To creates a
CLA instance:
uuid=$(uuidgen)
echo $uuid > /sys/class/mdev_bus/<pdev>/mdev_supported_types/arm-cla-pl1/create
cla_cpu=$(< /sys/bus/mdev/devices/$uuid/cpu)
This creates a user interface in /dev/vfio: a VFIO group in
/dev/vfio/<n>, discoverable by reading the symlink
/sys/bus/mdev/devices/$uuid/iommu_group. Although CLA does not use an
IOMMU, VFIO relies on IOMMU groups as an assignable unit. Here they
correspond to CLA domains, whose devices communicate through a private
bus.
The CLA device gets assigned exclusively to the first VMM that opens the
VFIO group and device. To launch a VM with a CLA attached to vCPU #4,
with kvmtool:
lkvm run -k Image --console virtio -c 8
--vfio-platform $uuid,type=arm-cla,vcpu=4
--vcpu-affinity 4=$cla_cpu # Bind vCPU 4 to the CLA's CPU
Opens the group, gets the device fd, mmaps the frame, and launches the
guest. The given vCPU thread is pinned to the CLA's CPU.
Pinning vCPUs is not guaranteed. If the host decides to power off the
CPUs with assigned CLAs, the guest application can't access the assigned
device anymore. It would now access a RAZ/WI memory region.
For DMA, because the TLB entries tagged with VMID+ASID are shared
between CPU and accelerator, we pin the VMID. The stage 2 page tables
are shared between CPU and CLA. CLA reports DMA page faults directly to
the user, who performs the access from the CPU to handle both stage 1
and stage 2 faults.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Kconfig | 16 +
drivers/misc/arm-cla/Makefile | 2 +
drivers/misc/arm-cla/arm-cla.h | 48 +++
drivers/misc/arm-cla/cla-init.c | 11 +-
drivers/misc/arm-cla/cla-mdev.c | 469 ++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 9 +-
6 files changed, 553 insertions(+), 2 deletions(-)
create mode 100644 drivers/misc/arm-cla/cla-mdev.c
diff --git a/drivers/misc/arm-cla/Kconfig b/drivers/misc/arm-cla/Kconfig
index f8436a12c1e6..5def727b7a34 100644
--- a/drivers/misc/arm-cla/Kconfig
+++ b/drivers/misc/arm-cla/Kconfig
@@ -8,3 +8,19 @@ config ARM_CLA
accelerators. It is only accessible from its associated CPU.
If unsure, say N.
+
+config ARM_CLA_MDEV
+ bool "Mediated device for Arm CLA"
+ default y
+ depends on KVM
+ depends on VFIO
+ depends on ARM_CLA
+ depends on VFIO_GROUP
+ select VFIO_MDEV
+ help
+ Enable the VFIO mediated device interface for assigning Arm Core Local
+ Accelerators (CLAs) to KVM guests. Guests can access the CLA PL0 and
+ PL1 registers, while the host controls stage 2 address translation.
+ Devices in the same CLA domain can only be assigned to the same guest.
+
+ If unsure, say N.
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index f848c608e8f1..973390d0b428 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -11,3 +11,5 @@ arm-cla-y := \
cla-user.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
+
+arm-cla-$(CONFIG_ARM_CLA_MDEV) += cla-mdev.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 43a58a329525..93365f525abb 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -14,6 +14,7 @@
#include <linux/kref.h>
#include <linux/kthread.h>
#include <linux/list.h>
+#include <linux/mdev.h>
#include <linux/mm.h>
#include <linux/mm_types.h>
#include <linux/mutex.h>
@@ -51,6 +52,11 @@ struct cla_accel_desc {
u64 revidr;
};
+enum cla_mdev_type {
+ CLA_MDEV_TYPE_PL1,
+ CLA_MDEV_NUM_TYPES,
+};
+
/**
* struct cla_dev - CLA device
*
@@ -71,6 +77,11 @@ struct cla_accel_desc {
* @broken: Hardware failure.
* @worker: CPU-bound worker to communicate with CLA.
* @call: Scheduling work.
+ *
+ * Other mutable state:
+ * @mdev_parent: Mdev structure for the VFIO interface.
+ * @mdev_types: Mdev types that can be created from this device.
+ * @mdev_types_ptr: Pointers to @mdev_type.
*/
struct cla_dev {
unsigned int cpu;
@@ -88,6 +99,12 @@ struct cla_dev {
bool broken;
struct kthread_worker *worker;
struct cla_call_on_cpu call;
+
+#if IS_ENABLED(CONFIG_ARM_CLA_MDEV)
+ struct mdev_parent mdev_parent;
+ struct mdev_type mdev_types[CLA_MDEV_NUM_TYPES];
+ struct mdev_type *mdev_types_ptr[CLA_MDEV_NUM_TYPES];
+#endif
};
/**
@@ -100,6 +117,7 @@ struct cla_dev {
* @pg_offset: Mmap offset of the first device.
* @nr_devs: Number of devices in the domain.
* @devs: Devices.
+ * @iommu_group: Handle for VFIO group.
*
* Mutable, only accessed under @lock:
* @lock: Protects the following members.
@@ -127,6 +145,10 @@ struct cla_domain {
struct cla_ctx *assigned_ctx;
struct kthread_worker *worker;
struct kthread_delayed_work reassign;
+
+#if IS_ENABLED(CONFIG_ARM_CLA_MDEV)
+ struct iommu_group *iommu_group;
+#endif
};
/**
@@ -187,6 +209,12 @@ struct cla_ctx_key {
.file = (_file), \
}
+#define CLA_CTX_KEY_PL1(_kvm) \
+ { \
+ .type = CLA_CTX_PL1, \
+ .kvm = (_kvm), \
+ }
+
/**
* struct cla_ctx - Domain context
*
@@ -342,4 +370,24 @@ int cla_user_init(void);
void cla_user_exit(void);
u8 cla_user_accels(struct cla_dev *dev, struct file *file);
+#if IS_ENABLED(CONFIG_ARM_CLA_MDEV)
+int cla_mdev_init(void);
+void cla_mdev_exit(void);
+void cla_mdev_enable(void);
+int cla_domain_mdev_init(struct cla_domain *domain);
+void cla_domain_mdev_exit(struct cla_domain *domain);
+#else
+static inline int cla_mdev_init(void)
+{
+ return 0;
+}
+static inline void cla_mdev_exit(void) {}
+static inline void cla_mdev_enable(void) {}
+static inline int cla_domain_mdev_init(struct cla_domain *domain)
+{
+ return 0;
+}
+static inline void cla_domain_mdev_exit(struct cla_domain *domain) {}
+#endif
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 1afe20aa877f..a0067240809a 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -393,6 +393,7 @@ static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
dev->regs = cla_get_regs(regs, cla_kernel_pl);
dev->cpu = cpu;
dev->dev = parent;
+ dev_set_drvdata(parent, dev);
kthread_init_work(&dev->call.switch_ctx, cla_dev_switch_ctx_work);
@@ -519,13 +520,20 @@ static int __init cla_module_init(void)
cla_cpuhp_state = ret;
}
+ ret = cla_mdev_init();
+ if (ret)
+ goto err_hardware;
+
/* Must be last fallible step; see cla_user_init(). */
ret = cla_user_init();
if (ret)
- goto err_hardware;
+ goto err_mdev_exit;
+ cla_mdev_enable();
return 0;
+err_mdev_exit:
+ cla_mdev_exit();
err_hardware:
cla_domains_sched_exit();
if (cla_cpuhp_state >= 0)
@@ -539,6 +547,7 @@ static int __init cla_module_init(void)
static void __exit cla_module_exit(void)
{
cla_user_exit();
+ cla_mdev_exit();
cla_domains_sched_exit();
cpuhp_remove_state(cla_cpuhp_state);
cla_domains_free();
diff --git a/drivers/misc/arm-cla/cla-mdev.c b/drivers/misc/arm-cla/cla-mdev.c
new file mode 100644
index 000000000000..66c5dd197542
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-mdev.c
@@ -0,0 +1,469 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - vfio-mdev interface
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/cpuhplock.h>
+#include <linux/vfio.h>
+#include <linux/mdev.h>
+
+#include "arm-cla.h"
+
+struct cla_mdev {
+ struct vfio_device vdev;
+ struct cla_dev *dev;
+ enum cla_mdev_type type;
+};
+
+static bool cla_mdev_ready;
+
+/*
+ * Logical CPU number. After mdev creation, this attribute appears in
+ * /sys/bus/mdev/devices/$uuid/cpu
+ *
+ * This information is crucial to the user, because the device is only
+ * accessible from this CPU.
+ */
+static ssize_t cpu_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct cla_mdev *mdev = dev_get_drvdata(dev);
+
+ return sysfs_emit(buf, "%u\n", mdev->dev->cpu);
+}
+DEVICE_ATTR_RO(cpu);
+
+static const struct attribute *const cla_mdev_sysfs_attrs[] = {
+ &dev_attr_cpu.attr, /* CPU ID */
+ NULL,
+};
+ATTRIBUTE_GROUPS(cla_mdev_sysfs);
+
+static const struct mdev_type cla_mdev_types[CLA_MDEV_NUM_TYPES] = {
+ /*
+ * Assign the PL0 and PL1 registers, with PL2 controlled by the
+ * kernel (or hypervisor)
+ */
+ [CLA_MDEV_TYPE_PL1] = {
+ .sysfs_name = "pl1",
+ .pretty_name = "CLA PL1 and PL0 registers exclusively owned",
+ },
+};
+
+static int cla_mdev_switch_ctx(struct cla_dev *dev, struct cla_ctx *prev,
+ struct cla_ctx *next)
+{
+ struct cla_call_on_cpu *call = &dev->call;
+
+ lockdep_assert_cpus_held();
+
+ /*
+ * No race with reassignment work because we're holding the domain lock
+ * and we're domain->assigned_ctx.
+ */
+ call->sched.prev_ctx = prev;
+ call->sched.next_ctx = next;
+ call->ret = 0;
+ if (dev->worker && !dev->broken)
+ kthread_queue_work(dev->worker, &call->switch_ctx);
+
+ kthread_flush_work(&call->switch_ctx);
+
+ return call->ret;
+}
+
+/*
+ * Perform a RESET on all accelerators for this dev and clear the CLA
+ * regs. The VMM calls this when rebooting a VM. It doesn't need to be
+ * efficient. Other devices in the domain are not reset.
+ */
+static int cla_mdev_reset(struct cla_mdev *mdev)
+{
+ int ret = 0;
+ struct cla_ctx *ctx;
+ struct cla_dev *dev = mdev->dev;
+ struct cla_domain *domain = dev->domain;
+
+ /* Hotplug callbacks may take domain->lock, so acquire this first. */
+ cpus_read_lock();
+ mutex_lock(&domain->lock);
+ if (domain->broken) {
+ ret = -EIO;
+ goto out_unlock;
+ }
+
+ ctx = domain->assigned_ctx;
+ /*
+ * This function is only reachable when a valid PL1 ctx corresponding to
+ * the VM is assigned to the domain: cla_mdev_open_device() waits for
+ * the ctx to be assigned, and the ctx only gets deassigned when the
+ * file has no reference anymore (no ioctl possible). So the following
+ * are all bugs.
+ */
+ if (WARN_ON(!ctx) ||
+ WARN_ON(!cla_ctx_is_pl1(ctx)) ||
+ WARN_ON(ctx->key.kvm != mdev->vdev.kvm)) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
+ /*
+ * This provides a clean config. The stage 1 MTC is clear. The
+ * accelerators and CLA regs are reset.
+ */
+ ret = cla_mdev_switch_ctx(dev, ctx, NULL);
+ if (ret)
+ goto out_unlock;
+
+ ret = cla_mdev_switch_ctx(dev, NULL, ctx);
+out_unlock:
+ if (ret)
+ cla_domain_set_broken(domain);
+ mutex_unlock(&domain->lock);
+ cpus_read_unlock();
+ return ret;
+}
+
+static int cla_mdev_open_device(struct vfio_device *vdev)
+{
+ struct cla_mdev *mdev =
+ container_of(vdev, struct cla_mdev, vdev);
+ struct cla_domain *domain = mdev->dev->domain;
+ struct cla_ctx_key key = CLA_CTX_KEY_PL1(vdev->kvm);
+ struct cla_ctx *ctx;
+ int ret;
+
+ /*
+ * The user registers a KVM instance with this VFIO group. We use it to
+ * program the stage-2 MTC. stage-1 MTC is managed by the user/guest.
+ * KVM, if present, is valid for the lifetime of the vfio device file.
+ */
+ if (!vdev->kvm)
+ return -EINVAL;
+
+ /*
+ * Obtain a context for exclusive assignment. If other contexts exist
+ * for this domain, this returns an error. Other devices assigned to the
+ * same VM within the same domain get the same context.
+ */
+ ctx = cla_ctx_create(domain, &key);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ mutex_lock(&domain->lock);
+ if (domain->broken) {
+ ret = -EIO;
+ goto err_destroy;
+ }
+
+ /* Not assigned and not queued, assign it. */
+ if (domain->assigned_ctx != ctx && list_empty(&ctx->queue_node)) {
+ list_add_tail(&ctx->queue_node, &domain->queued_ctxs);
+ cla_domain_schedule_reassignment(domain);
+ }
+ mutex_unlock(&domain->lock);
+
+ ret = wait_event_interruptible(ctx->waitq,
+ READ_ONCE(domain->assigned_ctx) == ctx ||
+ READ_ONCE(domain->broken));
+
+ mutex_lock(&domain->lock);
+ /*
+ * At this point the context is either assigned or still in the queue
+ * (if we were interrupted). In case of error, cla_ctx_destroy() kills the
+ * context and deassigns it.
+ */
+ if (ret)
+ goto err_destroy;
+
+ if (domain->broken) {
+ ret = -EIO;
+ goto err_destroy;
+ }
+ mutex_unlock(&domain->lock);
+ return 0;
+
+err_destroy:
+ mutex_unlock(&domain->lock);
+ cla_ctx_destroy(domain, &key);
+ return ret;
+}
+
+static void cla_mdev_close_device(struct vfio_device *vdev)
+{
+ struct cla_mdev *mdev =
+ container_of(vdev, struct cla_mdev, vdev);
+ struct cla_ctx_key key = CLA_CTX_KEY_PL1(vdev->kvm);
+
+ cla_ctx_destroy(mdev->dev->domain, &key);
+}
+
+static long cla_mdev_ioctl(struct vfio_device *vdev, unsigned int cmd,
+ unsigned long arg)
+{
+ struct cla_mdev *mdev =
+ container_of(vdev, struct cla_mdev, vdev);
+ unsigned long minsz;
+
+ switch (cmd) {
+ case VFIO_DEVICE_GET_INFO:
+ {
+ struct vfio_device_info info;
+
+ minsz = offsetofend(struct vfio_device_info, num_irqs);
+
+ if (copy_from_user(&info, (void __user *)arg, minsz))
+ return -EFAULT;
+
+ if (info.argsz < minsz)
+ return -EINVAL;
+
+ info.flags = VFIO_DEVICE_FLAGS_PLATFORM |
+ VFIO_DEVICE_FLAGS_RESET;
+ info.num_regions = 1;
+ info.num_irqs = 0;
+
+ if (copy_to_user((void __user *)arg, &info, minsz))
+ return -EFAULT;
+
+ return 0;
+ }
+ case VFIO_DEVICE_RESET:
+ return cla_mdev_reset(mdev);
+ case VFIO_DEVICE_GET_IRQ_INFO:
+ case VFIO_DEVICE_SET_IRQS:
+ default:
+ return -EINVAL;
+ }
+ return -ENOTTY;
+}
+
+static int cla_mdev_get_region_info_caps(struct vfio_device *vdev,
+ struct vfio_region_info *region_info,
+ struct vfio_info_cap *caps)
+{
+ if (region_info->index != 0)
+ return -EINVAL;
+
+ region_info->offset = 0;
+ region_info->size = 2 * CLA_REG_SIZE;
+ region_info->flags = VFIO_REGION_INFO_FLAG_MMAP |
+ VFIO_REGION_INFO_FLAG_READ |
+ VFIO_REGION_INFO_FLAG_WRITE;
+ return 0;
+}
+
+/* Mmap PL0 + PL1 frames. */
+static int cla_mdev_mmap(struct vfio_device *vdev, struct vm_area_struct *vma)
+{
+ struct cla_mdev *mdev =
+ container_of(vdev, struct cla_mdev, vdev);
+ unsigned long vm_flags_required = VM_SHARED | VM_READ | VM_WRITE;
+ size_t map_size = vma->vm_end - vma->vm_start;
+ size_t region_size = 2 * CLA_REG_SIZE;
+
+ if (vma->vm_pgoff != 0)
+ return -EINVAL;
+
+ if (vma->vm_end < vma->vm_start)
+ return -EINVAL;
+ if (map_size != region_size)
+ return -EINVAL;
+ if ((vma->vm_flags & vm_flags_required) != vm_flags_required)
+ return -EINVAL;
+ if (vma->vm_flags & VM_EXEC)
+ return -EINVAL;
+
+ vm_flags_mod(vma, VM_DONTCOPY | VM_DONTDUMP | VM_DONTEXPAND |
+ VM_IO | VM_PFNMAP, VM_MAYEXEC);
+ vma->vm_page_prot = pgprot_device(vma->vm_page_prot);
+
+ return remap_pfn_range(vma, vma->vm_start, mdev->dev->pfn, map_size,
+ vma->vm_page_prot);
+}
+
+static const struct vfio_device_ops cla_mdev_pl1_ops = {
+ .name = "vfio-arm-cla-pl1",
+ .open_device = cla_mdev_open_device,
+ .close_device = cla_mdev_close_device,
+ .ioctl = cla_mdev_ioctl,
+ .get_region_info_caps = cla_mdev_get_region_info_caps,
+ .mmap = cla_mdev_mmap,
+};
+
+/*
+ * Admin creates an mdev device
+ *
+ * uuid=$(uuidgen)
+ * echo $uuid > /sys/class/mdev_bus/<pdev>/mdev_supported_types/arm-cla-pl1/create
+ */
+static int cla_mdev_probe(struct mdev_device *mdev)
+{
+ int ret = -ENODEV;
+ struct cla_dev *dev;
+ enum cla_mdev_type type;
+ struct cla_mdev *cla_mdev;
+
+ /* Ensure the module isn't being removed */
+ if (!try_module_get(THIS_MODULE))
+ return -ENODEV;
+
+ /* Ensure the module has finished initializing */
+ if (!smp_load_acquire(&cla_mdev_ready))
+ goto err_put_module;
+
+ dev = dev_get_drvdata(mdev->dev.parent);
+ type = mdev->type - dev->mdev_types;
+
+ ret = -EINVAL;
+ if (WARN_ON(type < 0 || type >= CLA_MDEV_NUM_TYPES))
+ goto err_put_module;
+
+ cla_mdev = vfio_alloc_device(cla_mdev, vdev, &mdev->dev,
+ &cla_mdev_pl1_ops);
+ if (IS_ERR(cla_mdev)) {
+ ret = PTR_ERR(cla_mdev);
+ goto err_put_module;
+ }
+
+ cla_mdev->dev = dev;
+ cla_mdev->type = type;
+
+ dev_set_drvdata(&mdev->dev, cla_mdev);
+
+ ret = iommu_group_add_device(dev->domain->iommu_group, &mdev->dev);
+ if (ret)
+ goto err_put_vdev;
+
+ ret = vfio_register_cpu_mmu_dev(&cla_mdev->vdev);
+ if (ret)
+ goto err_put_group;
+
+ return 0;
+
+err_put_group:
+ iommu_group_remove_device(&mdev->dev);
+err_put_vdev:
+ dev_set_drvdata(&mdev->dev, NULL);
+ vfio_put_device(&cla_mdev->vdev);
+err_put_module:
+ module_put(THIS_MODULE);
+ return ret;
+}
+
+/*
+ * Admin removes an mdev device
+ *
+ * echo 1 > /sys/bus/mdev/devices/$uuid/remove
+ */
+static void cla_mdev_remove(struct mdev_device *mdev)
+{
+ struct cla_mdev *cla_mdev = dev_get_drvdata(&mdev->dev);
+
+ vfio_unregister_group_dev(&cla_mdev->vdev);
+ iommu_group_remove_device(&mdev->dev);
+ dev_set_drvdata(&mdev->dev, NULL);
+ vfio_put_device(&cla_mdev->vdev);
+ module_put(THIS_MODULE);
+}
+
+static struct mdev_driver cla_mdev_driver = {
+ .device_api = VFIO_DEVICE_API_PLATFORM_STRING,
+ .max_instances = 1,
+ .driver = {
+ .name = "arm-cla-mdev",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = cla_mdev_sysfs_groups,
+ },
+ .probe = cla_mdev_probe,
+ .remove = cla_mdev_remove,
+};
+
+/* Allocate IOMMU groups so VFIO can create its groups based on CLA domains */
+int cla_domain_mdev_init(struct cla_domain *domain)
+{
+ struct iommu_group *iommu_group;
+
+ if (cla_kernel_pl < 2)
+ return 0;
+
+ iommu_group = iommu_group_alloc();
+ if (IS_ERR(iommu_group))
+ return PTR_ERR(iommu_group);
+
+ domain->iommu_group = iommu_group;
+ return 0;
+}
+
+void cla_domain_mdev_exit(struct cla_domain *domain)
+{
+ iommu_group_put(domain->iommu_group);
+}
+
+int __init cla_mdev_init(void)
+{
+ int i, j;
+ int ret;
+ struct cla_dev *dev;
+
+ if (cla_kernel_pl < 2)
+ return 0;
+
+ ret = mdev_register_driver(&cla_mdev_driver);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < cla_nr_devs; i++) {
+ dev = cla_lut_pg[i];
+
+ /*
+ * mdev_type is a kobj in sysfs, so it cannot be shared between
+ * CLAs. Each CLA needs its own instance.
+ */
+ BUILD_BUG_ON(sizeof(cla_mdev_types) != sizeof(dev->mdev_types));
+ memcpy(dev->mdev_types, cla_mdev_types, sizeof(cla_mdev_types));
+ for (j = 0; j < CLA_MDEV_NUM_TYPES; j++)
+ dev->mdev_types_ptr[j] = &dev->mdev_types[j];
+
+ ret = mdev_register_parent(&dev->mdev_parent, dev->dev,
+ &cla_mdev_driver, dev->mdev_types_ptr,
+ CLA_MDEV_NUM_TYPES);
+ if (ret)
+ goto err_unregister;
+ }
+
+ return 0;
+
+err_unregister:
+ for (i--; i >= 0; i--) {
+ dev = cla_lut_pg[i];
+ mdev_unregister_parent(&dev->mdev_parent);
+ }
+ mdev_unregister_driver(&cla_mdev_driver);
+ return ret;
+}
+
+/* Enable the interface, once the module initialization is complete */
+void __init cla_mdev_enable(void)
+{
+ /* Synchronize with cla_mdev_probe() */
+ smp_store_release(&cla_mdev_ready, true);
+}
+
+void cla_mdev_exit(void)
+{
+ int i;
+
+ if (cla_kernel_pl < 2)
+ return;
+
+ for (i = 0; i < cla_nr_devs; i++) {
+ struct cla_dev *dev = cla_lut_pg[i];
+
+ mdev_unregister_parent(&dev->mdev_parent);
+ }
+ mdev_unregister_driver(&cla_mdev_driver);
+}
diff --git a/drivers/misc/arm-cla/cla-topology.c b/drivers/misc/arm-cla/cla-topology.c
index ed7acec70019..ead6c25c6391 100644
--- a/drivers/misc/arm-cla/cla-topology.c
+++ b/drivers/misc/arm-cla/cla-topology.c
@@ -51,12 +51,18 @@ static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int fwi
}
domain->devs[0] = dev;
- ret = cla_domain_sched_init(domain);
+ ret = cla_domain_mdev_init(domain);
if (ret)
goto err_free_devs;
+ ret = cla_domain_sched_init(domain);
+ if (ret)
+ goto err_exit_mdev;
+
return domain;
+err_exit_mdev:
+ cla_domain_mdev_exit(domain);
err_free_devs:
kfree(domain->devs);
err_free_id:
@@ -187,6 +193,7 @@ void cla_domains_free(void)
cla_lut_cpu = NULL;
cla_lut_pg = NULL;
xa_for_each(&cla_domains, fwid, domain) {
+ cla_domain_mdev_exit(domain);
xa_erase(&cla_domains, fwid);
kfree(domain->devs);
kfree(domain);
--
2.43.0
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [RFC PATCH v2 15/15] misc/arm-cla: Add vfio-mdev user interface
2026-10-15 14:25 ` [RFC PATCH v2 15/15] misc/arm-cla: Add vfio-mdev user interface Ryan Roberts
@ 2026-10-15 16:41 ` Jason Gunthorpe
0 siblings, 0 replies; 20+ messages in thread
From: Jason Gunthorpe @ 2026-10-15 16:41 UTC (permalink / raw)
To: Ryan Roberts
Cc: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Marc Zyngier, Oliver Upton, Alex Williamson,
Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet, linux-kernel,
linux-arm-kernel, kvmarm, kvm, devicetree, dri-devel, linux-doc
On Thu, Oct 15, 2026 at 03:25:49PM +0100, Ryan Roberts wrote:
> From: Jean-Philippe Brucker <jpb@kernel.org>
>
> Implement an interface to assign CLA devices to KVM guests. To creates a
> CLA instance:
>
> uuid=$(uuidgen)
> echo $uuid > /sys/class/mdev_bus/<pdev>/mdev_supported_types/arm-cla-pl1/create
> cla_cpu=$(< /sys/bus/mdev/devices/$uuid/cpu)
I think I said this last time, please don't use vfio mdev for
anything. There is nothing mediated about this.
Jason
^ permalink raw reply [flat|nested] 20+ messages in thread