* [PATCH v28 7/7] arm64: dts: imx8ulp-evk: add reserved memory property
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
Reserve 1MB of DDR memory region due to EdgeLock Enclave's hardware
limitation restricting access to DDR addresses from 0x80000000
to 0xafffffff.
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
---
arch/arm64/boot/dts/freescale/imx8ulp-evk.dts | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts b/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
index 5dea66c1e7aa..16399d921e04 100644
--- a/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
/*
- * Copyright 2021 NXP
+ * Copyright 2021, 2025 NXP
*/
/dts-v1/;
@@ -37,6 +37,12 @@ linux,cma {
linux,cma-default;
};
+ ele_reserved: memory@90000000 {
+ compatible = "shared-dma-pool";
+ reg = <0 0x90000000 0 0x100000>;
+ no-map;
+ };
+
m33_reserved: noncacheable-section@a8600000 {
reg = <0 0xa8600000 0 0x1000000>;
no-map;
@@ -259,6 +265,10 @@ &usdhc0 {
status = "okay";
};
+&hsm0 {
+ memory-region = <&ele_reserved>;
+};
+
&fec {
pinctrl-names = "default", "sleep";
pinctrl-0 = <&pinctrl_enet>;
--
2.43.0
^ permalink raw reply related
* [PATCH v28 6/7] arm64: dts: imx8ulp: add secure enclave node
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
Add support for NXP secure enclave called EdgeLock Enclave firmware (se-fw)
for imx8ulp-evk.
Add label sram0 for sram@2201f000 and add secure-enclave node
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
---
arch/arm64/boot/dts/freescale/imx8ulp.dtsi | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/boot/dts/freescale/imx8ulp.dtsi b/arch/arm64/boot/dts/freescale/imx8ulp.dtsi
index c6d1bb9edf38..182f1444b643 100644
--- a/arch/arm64/boot/dts/freescale/imx8ulp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8ulp.dtsi
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
/*
- * Copyright 2021 NXP
+ * Copyright 2021, 2025 NXP
*/
#include <dt-bindings/clock/imx8ulp-clock.h>
@@ -153,7 +153,7 @@ sosc: clock-sosc {
#clock-cells = <0>;
};
- sram@2201f000 {
+ sram0: sram@2201f000 {
compatible = "mmio-sram";
reg = <0x0 0x2201f000 0x0 0x1000>;
@@ -185,6 +185,13 @@ scmi_sensor: protocol@15 {
#thermal-sensor-cells = <1>;
};
};
+
+ hsm0: secure-enclave {
+ compatible = "fsl,imx8ulp-se-ele-hsm";
+ mbox-names = "tx", "rx";
+ mboxes = <&s4muap 0 0>, <&s4muap 1 0>;
+ sram = <&sram0>;
+ };
};
cm33: remoteproc-cm33 {
--
2.43.0
^ permalink raw reply related
* [PATCH v28 5/7] firmware: drivers: imx: adds miscdev
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel,
sashiko-bot
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
Adds the driver for communication interface to secure-enclave, that
enables exchanging messages with NXP secure enclave HW IP(s)
like EdgeLock Enclave, from:
- User-Space Applications via character driver.
ABI documentation for the NXP secure-enclave driver.
User-space library using this driver:
- i.MX Secure Enclave library:
-- URL: https://github.com/nxp-imx/imx-secure-enclave.git,
- i.MX Secure Middle-Ware:
-- URL: https://github.com/nxp-imx/imx-smw.git
Following checks are performed on the incoming msg-header,
to block exchanging invalid arbitrary commands:
- maximum allowed words,
- check if command-tag & response-tag are valid
- version,
- command id validation check, to allow limited base-line API(s)
and restrict following:
- exchanging power management commands.
- reset requests.
- BBSM configuration requests.
- re-initializing the FW.
- RNG init
- CAAM resource release management
- SE's internal memory management.
from user-space.
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
---
Changes from v27 to v28
Fix 10 of the 13 issues reported by Sashiko AI review
[High] se_ctrl.c: se_if_fops_read(): reacquire fops_lock after
ele_msg_rcv() returns before accessing pending lists or rx_msg.
fops_lock is dropped before the blocking wait; a concurrent close could
free the DMA buffers and pending lists while the read is blocked, leading
to UAF and list corruption. Re-check cleanup_done under fops_lock before
touching any shared state.
[High] ele_common.c: se_val_rsp_hdr_n_status(): guard msg->data[0] read
with if (header->size > SE_MU_HDR_WORD_SZ). A header-only response (1
word) is valid; the unconditional read caused a KASAN slab-out-of-bounds.
[High] ele_common.h: reduce SE_RCV_MSG_DEFAULT_TIMEOUT from 5000 s to
3000 ms. After a signal interrupts the interruptible wait, ele_msg_rcv()
switches to TASK_UNINTERRUPTIBLE. A 5000-second uninterruptible sleep
reliably triggers the hung-task watchdog. 3000 ms is well below the
default 120 s threshold.
[High] se_ctrl.c: cleanup_se_shared_mem(): call
se_dev_ctx_shared_mem_cleanup() to free se_buf_desc list entries before
releasing the DMA backing memory, fixing a leak when the fd is closed
with pending I/O buffers.
[High] se_ctrl.c: se_dev_ctx_shared_mem_cleanup(): skip memset of DMA
buffers when the context is the fw_busy one (command timed out). The
firmware may still be actively accessing the buffer; zeroing it would
corrupt the in-flight DMA transaction.
[High] se_ctrl.c: cleanup_se_shared_mem(): guard against calling
dma_free_coherent() with a NULL ptr (probe failure before DMA alloc
succeeded).
[High] The command-receiver rx_msg_sz update is now done under
cmd_receiver_clbk_hdl.clbk_rx_lock. The read path also validates
cmd_receiver_clbk_hdl.dev_ctx, rx_msg, and rx_msg_sz under the same callback
lock before consuming the message.
[High] se_ctrl.c: se_if_probe_cleanup(): call se_clear_fw_busy() before
cancel_work_sync(). A late mailbox interrupt arriving between the two
calls would see fw_busy still set and schedule work on the already-
cancelled fw_busy_work, causing a use-after-free.
[Medium] se_ctrl.c: cleanup_dev_ctx(): remove goto from inside a
scoped_guard() block. Use a local already_done flag instead to keep the
cleanup path flat and avoid confusing ownership semantics.
[Misc] se_ctrl.c: se_ctrl.h: add SE_MU_HDR_WORD_SZ = 1 constant.
Make se_is_fw_busy_ctx(), se_dev_ctx_shared_mem_cleanup() and
get_shared_mem_slot() non-static so they can be called from ele_common.c
and future callers.
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260715-imx-se-if-v27-0-bb7c45952f06@nxp.com?part=5
---
Documentation/ABI/testing/se-cdev | 44 ++
drivers/firmware/imx/Makefile | 2 +-
drivers/firmware/imx/ele_base_msg.c | 61 +-
drivers/firmware/imx/ele_base_msg.h | 19 +
drivers/firmware/imx/ele_common.c | 95 ++-
drivers/firmware/imx/ele_common.h | 10 +
drivers/firmware/imx/ele_fw_api.c | 57 ++
drivers/firmware/imx/ele_fw_api.h | 18 +
drivers/firmware/imx/se_ctrl.c | 1251 ++++++++++++++++++++++++++++++++++-
drivers/firmware/imx/se_ctrl.h | 74 ++-
include/uapi/linux/se_ioctl.h | 97 +++
11 files changed, 1675 insertions(+), 53 deletions(-)
diff --git a/Documentation/ABI/testing/se-cdev b/Documentation/ABI/testing/se-cdev
new file mode 100644
index 000000000000..c6b8e16bda78
--- /dev/null
+++ b/Documentation/ABI/testing/se-cdev
@@ -0,0 +1,44 @@
+What: /dev/<se>_mu[0-9]+_ch[0-9]+
+Date: Mar 2025
+KernelVersion: 6.8
+Contact: linux-imx@nxp.com, pankaj.gupta@nxp.com
+Description:
+ NXP offers multiple hardware IP(s) for secure enclaves like EdgeLock-
+ Enclave(ELE), SECO. The character device file descriptors
+ /dev/<se>_mu*_ch* are the interface between userspace NXP's secure-
+ enclave shared library and the kernel driver.
+
+ The ioctl(2)-based ABI is defined and documented in
+ [include]<linux/firmware/imx/ele_mu_ioctl.h>.
+ ioctl(s) are used primarily for:
+
+ - shared memory management
+ - allocation of I/O buffers
+ - getting mu info
+ - setting a dev-ctx as receiver to receive all the commands from FW
+ - getting SoC info
+ - send command and receive command response
+
+ The following file operations are supported:
+
+ open(2)
+ Currently the only useful flags are O_RDWR.
+
+ read(2)
+ Every read() from the opened character device context is waiting on
+ wait_event_interruptible, that gets set by the registered mailbox callback
+ function, indicating a message received from the firmware on message-
+ unit.
+
+ write(2)
+ Every write() to the opened character device context needs to acquire
+ mailbox_lock before sending message on to the message unit.
+
+ close(2)
+ Stops and frees up the I/O contexts that were associated
+ with the file descriptor.
+
+Users: https://github.com/nxp-imx/imx-secure-enclave.git,
+ https://github.com/nxp-imx/imx-smw.git,
+ crypto/skcipher,
+ drivers/nvmem/imx-ocotp-ele.c
diff --git a/drivers/firmware/imx/Makefile b/drivers/firmware/imx/Makefile
index 4412b15846b1..3f41131a0fdc 100644
--- a/drivers/firmware/imx/Makefile
+++ b/drivers/firmware/imx/Makefile
@@ -4,5 +4,5 @@ obj-$(CONFIG_IMX_SCU) += imx-scu.o misc.o imx-scu-irq.o rm.o imx-scu-soc.o
obj-${CONFIG_IMX_SCMI_CPU_DRV} += sm-cpu.o
obj-${CONFIG_IMX_SCMI_MISC_DRV} += sm-misc.o
obj-${CONFIG_IMX_SCMI_LMM_DRV} += sm-lmm.o
-sec_enclave-objs = se_ctrl.o ele_common.o ele_base_msg.o
+sec_enclave-objs = se_ctrl.o ele_common.o ele_base_msg.o ele_fw_api.o
obj-${CONFIG_IMX_SEC_ENCLAVE} += sec_enclave.o
diff --git a/drivers/firmware/imx/ele_base_msg.c b/drivers/firmware/imx/ele_base_msg.c
index 6a6db60887af..8f29f4a148d9 100644
--- a/drivers/firmware/imx/ele_base_msg.c
+++ b/drivers/firmware/imx/ele_base_msg.c
@@ -15,19 +15,48 @@
#define FW_DBG_DUMP_FIXED_STR "ELE"
+int ele_uapi_allowed_base_cmd(struct se_if_priv *priv,
+ struct se_msg_hdr *header)
+{
+ switch (header->command) {
+ case ELE_PING_REQ: return 0;
+ case ELE_DEBUG_DUMP_REQ: return 0;
+ case ELE_OEM_AUTH_CONTAINER_REQ: return 0;
+ case ELE_OEM_VERIFY_IMAGE_REQ: return 0;
+ case ELE_OEM_REL_CONTAINER_REQ: return 0;
+ case ELE_FW_LIFE_CYCLE_REQ: return 0;
+ case ELE_READ_FUSE_REQ: return 0;
+ case ELE_GET_FW_VERS_REQ: return 0;
+ case ELE_RETURN_LIFE_CYCLE_REQ: return 0;
+ case ELE_GET_EVENT_REQ: return 0;
+ case ELE_COMMIT_REQ: return 0;
+ case ELE_GEN_KEY_BLOB_REQ: return 0;
+ case ELE_GET_FW_STATUS_REQ: return 0;
+ case ELE_XIP_DECRYPT_REQ: return 0;
+ case ELE_WRITE_FUSE: return 0;
+ case ELE_GET_INFO_REQ: return 0;
+ case ELE_DEV_ATTEST_REQ: return 0;
+ case ELE_WRITE_SHADOW_FUSE_REQ: return 0;
+ case ELE_READ_SHADOW_FUSE_REQ: return 0;
+ default:
+ return -EACCES;
+ }
+}
+
static void ele_get_info_cleanup(struct se_if_priv *priv, u32 *buf, dma_addr_t d_addr,
size_t size)
{
if (priv->mem_pool)
gen_pool_free(priv->mem_pool, (unsigned long)buf, size);
- else
- dma_free_coherent(priv->dev, size, buf, d_addr);
+ else if (!se_is_fw_busy_ctx(priv->priv_dev_ctx))
+ se_dev_ctx_shared_mem_cleanup(priv->priv_dev_ctx);
}
int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info)
{
dma_addr_t get_info_addr = 0;
- u32 *get_info_data = NULL;
+ void *get_info_data = NULL;
+ u32 get_info_len;
int ret = 0;
if (!priv)
@@ -45,19 +74,19 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info)
if (!rx_msg)
return -ENOMEM;
- if (priv->mem_pool)
+ if (priv->mem_pool) {
get_info_data = gen_pool_dma_alloc(priv->mem_pool,
ELE_GET_INFO_BUFF_SZ,
&get_info_addr);
- else
- get_info_data = dma_alloc_coherent(priv->dev,
- ELE_GET_INFO_BUFF_SZ,
- &get_info_addr,
- GFP_KERNEL);
- if (!get_info_data) {
- dev_err(priv->dev,
- "%s: Failed to allocate get_info_addr.", __func__);
- return -ENOMEM;
+ } else {
+ get_info_len = ELE_GET_INFO_BUFF_SZ;
+ ret = get_shared_mem_slot(priv->priv_dev_ctx,
+ &get_info_len, &get_info_addr,
+ &get_info_data);
+ if (ret) {
+ dev_err(priv->dev, "Failed to allocate buffer.\n");
+ return -ENOMEM;
+ }
}
se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header,
@@ -70,20 +99,20 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info)
ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_GET_INFO_REQ_MSG_SZ,
rx_msg, ELE_GET_INFO_RSP_MSG_SZ);
if (ret < 0) {
- ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
+ ele_get_info_cleanup(priv, get_info_data, get_info_addr, get_info_len);
return ret;
}
ret = se_val_rsp_hdr_n_status(priv, rx_msg, ELE_GET_INFO_REQ,
ELE_GET_INFO_RSP_MSG_SZ, true);
if (ret < 0) {
- ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
+ ele_get_info_cleanup(priv, get_info_data, get_info_addr, get_info_len);
return ret;
}
memcpy(s_info, get_info_data, sizeof(*s_info));
- ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
+ ele_get_info_cleanup(priv, get_info_data, get_info_addr, get_info_len);
return ret;
}
diff --git a/drivers/firmware/imx/ele_base_msg.h b/drivers/firmware/imx/ele_base_msg.h
index 4c3699543e87..238465a4eccd 100644
--- a/drivers/firmware/imx/ele_base_msg.h
+++ b/drivers/firmware/imx/ele_base_msg.h
@@ -15,6 +15,23 @@
#define ELE_NONE_VAL 0x0
+#define ELE_OEM_AUTH_CONTAINER_REQ 0x87
+#define ELE_OEM_VERIFY_IMAGE_REQ 0x88
+#define ELE_OEM_REL_CONTAINER_REQ 0x89
+#define ELE_FW_LIFE_CYCLE_REQ 0x95
+#define ELE_READ_FUSE_REQ 0x97
+#define ELE_GET_FW_VERS_REQ 0x9d
+#define ELE_RETURN_LIFE_CYCLE_REQ 0xa0
+#define ELE_GET_EVENT_REQ 0xa2
+#define ELE_COMMIT_REQ 0xa8
+#define ELE_GEN_KEY_BLOB_REQ 0xaf
+#define ELE_GET_FW_STATUS_REQ 0xc5
+#define ELE_XIP_DECRYPT_REQ 0xc6
+#define ELE_WRITE_FUSE 0xd6
+#define ELE_DEV_ATTEST_REQ 0xdb
+#define ELE_WRITE_SHADOW_FUSE_REQ 0xf2
+#define ELE_READ_SHADOW_FUSE_REQ 0xf3
+
#define ELE_GET_INFO_REQ 0xda
#define ELE_GET_INFO_REQ_MSG_SZ 0x10
#define ELE_GET_INFO_RSP_MSG_SZ 0x08
@@ -95,4 +112,6 @@ int ele_service_swap(struct se_if_priv *priv, dma_addr_t addr,
int ele_fw_authenticate(struct se_if_priv *priv, dma_addr_t contnr_addr,
dma_addr_t img_addr);
int ele_debug_dump(struct se_if_priv *priv);
+int ele_uapi_allowed_base_cmd(struct se_if_priv *priv,
+ struct se_msg_hdr *header);
#endif
diff --git a/drivers/firmware/imx/ele_common.c b/drivers/firmware/imx/ele_common.c
index 1ca22c53b55c..71b17e767a6d 100644
--- a/drivers/firmware/imx/ele_common.c
+++ b/drivers/firmware/imx/ele_common.c
@@ -5,6 +5,27 @@
#include "ele_base_msg.h"
#include "ele_common.h"
+#include "ele_fw_api.h"
+#include "se_ctrl.h"
+
+int se_chk_tx_msg_hdr(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+
+ if (!header->size || header->size > MAX_WORD_SIZE)
+ return -EINVAL;
+
+ if (header->tag != priv->if_defs->cmd_tag &&
+ header->tag != priv->if_defs->rsp_tag)
+ return -EINVAL;
+
+ if (header->ver == priv->if_defs->base_api_ver)
+ return ele_uapi_allowed_base_cmd(priv, header);
+ else if (header->ver == priv->if_defs->fw_api_ver)
+ return ele_uapi_allowed_fw_cmd(dev_ctx, header);
+
+ return -EINVAL;
+}
/*
* se_update_msg_chksum() - calculate and update message checksum word.
@@ -46,16 +67,50 @@ int se_update_msg_chksum(u32 *msg, u32 msg_len)
return 0;
}
+static void se_mark_fw_busy(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->fw_busy_lock, flags);
+ if (!priv->fw_busy_dev_ctx) {
+ kref_get(&dev_ctx->refcount);
+ priv->fw_busy_dev_ctx = dev_ctx;
+ atomic_set(&priv->fw_busy, 1);
+ }
+ spin_unlock_irqrestore(&priv->fw_busy_lock, flags);
+}
+
+void set_se_rcv_msg_timeout(struct se_if_device_ctx *dev_ctx, u32 timeout_ms)
+{
+ dev_ctx->rcv_msg_timeout_jiffies = msecs_to_jiffies(timeout_ms);
+}
+
int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk_hdl)
{
struct se_if_priv *priv = dev_ctx->priv;
+ bool is_rsp_wait_with_timeout = false;
bool wait_uninterruptible = false;
unsigned long remaining_jiffies;
+ unsigned long deadline_jiffies;
unsigned long flags;
int ret;
- remaining_jiffies = MAX_SCHEDULE_TIMEOUT;
+ remaining_jiffies = dev_ctx->rcv_msg_timeout_jiffies;
+ if (se_clbk_hdl == &priv->waiting_rsp_clbk_hdl) {
+ is_rsp_wait_with_timeout = true;
+ deadline_jiffies = jiffies + remaining_jiffies;
+ }
+
do {
+ if (is_rsp_wait_with_timeout) {
+ if (time_after_eq(jiffies, deadline_jiffies)) {
+ ret = -ETIMEDOUT;
+ break;
+ }
+ remaining_jiffies = deadline_jiffies - jiffies;
+ }
+
if (wait_uninterruptible)
ret = wait_for_completion_timeout(&se_clbk_hdl->done,
remaining_jiffies);
@@ -70,7 +125,7 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk
* after the protocol transaction is brought back to a
* synchronized state.
*/
- if (se_clbk_hdl == &priv->waiting_rsp_clbk_hdl &&
+ if (is_rsp_wait_with_timeout &&
READ_ONCE(se_clbk_hdl->rx_msg)) {
WRITE_ONCE(se_clbk_hdl->signal_rcvd, true);
wait_uninterruptible = true;
@@ -96,7 +151,7 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk
spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
se_clbk_hdl->rx_msg = NULL;
if (!completion_done(&se_clbk_hdl->done))
- atomic_set(&priv->fw_busy, 1);
+ se_mark_fw_busy(dev_ctx);
spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
ret = -ETIMEDOUT;
@@ -162,7 +217,6 @@ int ele_msg_send_rcv(struct se_if_device_ctx *dev_ctx, void *tx_msg,
int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz)
{
struct se_if_priv *priv = dev_ctx->priv;
- unsigned long flags;
int err;
guard(mutex)(&priv->se_if_cmd_lock);
@@ -225,6 +279,7 @@ static bool check_hdr_exception_for_sz(struct se_if_priv *priv,
void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
{
struct se_clbk_handle *se_clbk_hdl;
+ bool schedule_fw_busy_work = false;
struct device *dev = mbox_cl->dev;
struct se_msg_hdr *header;
bool sz_mismatch = false;
@@ -304,9 +359,13 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
exp_rx_msg_sz = se_clbk_hdl->rx_msg_sz;
spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
if (!se_clbk_hdl->rx_msg) {
- /* Close circuit breaker on spinlock race */
- atomic_set(&priv->fw_busy, 0);
+ if (atomic_read(&priv->fw_busy))
+ schedule_fw_busy_work = true;
spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+
+ if (schedule_fw_busy_work)
+ schedule_work(&priv->fw_busy_work);
+
dev_info(dev, "ELE responded (late), recovery FW available.");
return;
}
@@ -348,41 +407,43 @@ int se_val_rsp_hdr_n_status(struct se_if_priv *priv, struct se_api_msg *msg,
u32 status;
if (header->tag != priv->if_defs->rsp_tag) {
- dev_err(priv->dev, "MSG[0x%x] Hdr: Resp tag mismatch. (0x%x != 0x%x)",
+ dev_dbg(priv->dev, "MSG[0x%x] Hdr: Resp tag mismatch. (0x%x != 0x%x)",
msg_id, header->tag, priv->if_defs->rsp_tag);
return -EINVAL;
}
if (header->command != msg_id) {
- dev_err(priv->dev, "MSG Header: Cmd id mismatch. (0x%x != 0x%x)",
+ dev_dbg(priv->dev, "MSG Header: Cmd id mismatch. (0x%x != 0x%x)",
header->command, msg_id);
return -EINVAL;
}
if ((sz % 4) || (header->size != (sz >> 2) &&
!check_hdr_exception_for_sz(priv, header))) {
- dev_err(priv->dev, "MSG[0x%x] Hdr: Cmd size mismatch. (0x%x != 0x%x)",
+ dev_dbg(priv->dev, "MSG[0x%x] Hdr: Cmd size mismatch. (0x%x != 0x%x)",
msg_id, header->size, (sz >> 2));
return -EINVAL;
}
if (is_base_api && header->ver != priv->if_defs->base_api_ver) {
- dev_err(priv->dev,
+ dev_dbg(priv->dev,
"MSG[0x%x] Hdr: Base API Vers mismatch. (0x%x != 0x%x)",
msg_id, header->ver, priv->if_defs->base_api_ver);
return -EINVAL;
} else if (!is_base_api && header->ver != priv->if_defs->fw_api_ver) {
- dev_err(priv->dev,
+ dev_dbg(priv->dev,
"MSG[0x%x] Hdr: FW API Vers mismatch. (0x%x != 0x%x)",
msg_id, header->ver, priv->if_defs->fw_api_ver);
return -EINVAL;
}
- status = RES_STATUS(msg->data[0]);
- if (status != priv->if_defs->success_tag) {
- dev_err(priv->dev, "Command Id[%x], Response Failure = 0x%x",
- header->command, status);
- return -EPERM;
+ if (header->size > SE_MU_HDR_WORD_SZ) {
+ status = RES_STATUS(msg->data[0]);
+ if (status != priv->if_defs->success_tag) {
+ dev_dbg(priv->dev, "Command Id[%x], Response Failure = 0x%x",
+ header->command, status);
+ return -EPERM;
+ }
}
return 0;
@@ -445,7 +506,7 @@ int se_restore_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem)
* restore imem state.
*/
if (s_info.d_addn_info.imem_state != ELE_IMEM_STATE_BAD || !imem->size)
- return -EIO;
+ return 0;
/*
* IMPORT command will restore IMEM from the given
diff --git a/drivers/firmware/imx/ele_common.h b/drivers/firmware/imx/ele_common.h
index f7d7e3e62dd4..f3b2bbb405ce 100644
--- a/drivers/firmware/imx/ele_common.h
+++ b/drivers/firmware/imx/ele_common.h
@@ -12,6 +12,15 @@
#define IMX_ELE_FW_DIR "imx/ele/"
+#define MAX_WORD_SIZE 0x20
+/* Default command-response timeout in ms. Kept well below the hung-task
+ * watchdog threshold (120 s by default) so a signal-interrupted wait that
+ * switches to TASK_UNINTERRUPTIBLE cannot trigger a hung-task panic.
+ */
+#define SE_RCV_MSG_DEFAULT_TIMEOUT 3000
+#define SE_RCV_MSG_LONG_TIMEOUT 5000000
+
+void set_se_rcv_msg_timeout(struct se_if_device_ctx *dev_ctx, u32 val);
int se_update_msg_chksum(u32 *msg, u32 msg_len);
int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk_hdl);
@@ -40,4 +49,5 @@ int se_save_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem);
int se_restore_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem);
+int se_chk_tx_msg_hdr(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header);
#endif /*__ELE_COMMON_H__ */
diff --git a/drivers/firmware/imx/ele_fw_api.c b/drivers/firmware/imx/ele_fw_api.c
new file mode 100644
index 000000000000..4f0f2cb53857
--- /dev/null
+++ b/drivers/firmware/imx/ele_fw_api.c
@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2026 NXP
+ */
+
+#include "se_ctrl.h"
+#include "ele_fw_api.h"
+
+static bool se_cmd_receiver_allowed_cmd(u8 cmd)
+{
+ switch (cmd) {
+ case ELE_SESSION_CLOSE_REQ:
+ case ELE_STORAGE_CLOSE_REQ:
+ case ELE_STORAGE_MASTER_IMPORT_REQ:
+ return true;
+ default:
+ return false;
+ }
+}
+
+int ele_uapi_allowed_fw_cmd(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+ bool is_cmd_receiver = false;
+
+ scoped_guard(mutex, &priv->modify_lock)
+ if (dev_ctx == priv->cmd_receiver_clbk_hdl.dev_ctx)
+ is_cmd_receiver = true;
+
+ if (header->tag == priv->if_defs->cmd_tag) {
+ if (is_cmd_receiver && !se_cmd_receiver_allowed_cmd(header->command))
+ return -EOPNOTSUPP;
+ }
+
+ if (header->tag == priv->if_defs->rsp_tag && !is_cmd_receiver)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+void fw_api_specific_ops(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header)
+{
+ if (header->command == ELE_STORAGE_OPEN_REQ) {
+ int rc = 0;
+
+ rc = set_dev_ctx_as_command_receiver(dev_ctx);
+ if (rc)
+ dev_err(dev_ctx->priv->dev,
+ "Failed to register %s as CMD-Receiver: %d\n",
+ dev_ctx->devname, rc);
+ }
+ if (header->command == ELE_STORAGE_CLOSE_REQ) {
+ scoped_guard(mutex, &dev_ctx->priv->modify_lock)
+ unset_dev_ctx_as_command_receiver(dev_ctx);
+ }
+}
+
diff --git a/drivers/firmware/imx/ele_fw_api.h b/drivers/firmware/imx/ele_fw_api.h
new file mode 100644
index 000000000000..e017359b5053
--- /dev/null
+++ b/drivers/firmware/imx/ele_fw_api.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2026 NXP
+ */
+
+#ifndef ELE_FW_API_H
+#define ELE_FW_API_H
+#include "se_ctrl.h"
+
+#define ELE_SESSION_OPEN_REQ 0x10u
+#define ELE_SESSION_CLOSE_REQ 0x11u
+#define ELE_STORAGE_OPEN_REQ 0xE0u
+#define ELE_STORAGE_CLOSE_REQ 0xE1u
+#define ELE_STORAGE_MASTER_IMPORT_REQ 0xE2u
+
+int ele_uapi_allowed_fw_cmd(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header);
+void fw_api_specific_ops(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header);
+#endif /* ELE_FW_API_H */
diff --git a/drivers/firmware/imx/se_ctrl.c b/drivers/firmware/imx/se_ctrl.c
index f43d06ab123b..a28e843eeec9 100644
--- a/drivers/firmware/imx/se_ctrl.c
+++ b/drivers/firmware/imx/se_ctrl.c
@@ -4,10 +4,10 @@
*/
#include <linux/bitfield.h>
+#include <linux/cleanup.h>
#include <linux/completion.h>
#include <linux/delay.h>
#include <linux/dev_printk.h>
-#include <linux/dma-direct.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
#include <linux/export.h>
@@ -16,6 +16,7 @@
#include <linux/genalloc.h>
#include <linux/init.h>
#include <linux/io.h>
+#include <linux/kref.h>
#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/of_platform.h>
@@ -24,22 +25,21 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/sys_soc.h>
+#include <uapi/linux/se_ioctl.h>
#include "ele_base_msg.h"
#include "ele_common.h"
+#include "ele_fw_api.h"
#include "se_ctrl.h"
+/* Maximum response buffer size in bytes for debug-dump replies. */
+#define MAX_ALLOWED_RX_MSG_SZ ELE_DEBUG_DUMP_RSP_SZ
+#define MAX_ALLOWED_TX_MSG_SZ SZ_4K
+
#define MAX_SOC_INFO_DATA_SZ 256
#define MBOX_TX_NAME "tx"
#define MBOX_RX_NAME "rx"
-#define SE_TYPE_STR_DBG "dbg"
-#define SE_TYPE_STR_HSM "hsm"
-
-#define SE_TYPE_ID_DBG 0x1
-
-#define SE_TYPE_ID_HSM 0x2
-
struct se_var_info {
u16 soc_rev;
/* To serialize populating common SoC level info. */
@@ -125,6 +125,13 @@ char *get_se_if_name(u8 se_if_id)
return NULL;
}
+static u32 get_se_soc_id(struct se_if_priv *priv)
+{
+ const struct se_if_node *if_node = device_get_match_data(priv->dev);
+
+ return if_node->se_info->soc_id;
+}
+
static struct se_fw_load_info *get_load_fw_instance(struct se_if_priv *priv)
{
return &priv->load_fw;
@@ -209,11 +216,264 @@ static int get_se_soc_info(struct se_if_priv *priv, const struct se_soc_info *se
return 0;
}
+static int load_firmware(struct se_if_priv *priv, const u8 *se_img_file_to_load)
+{
+ const struct firmware *fw = NULL;
+ dma_addr_t se_fw_dma_addr;
+ u32 se_fw_buf_len;
+ void *se_fw_buf;
+ int ret;
+
+ if (!se_img_file_to_load) {
+ dev_err(priv->dev, "FW image is not provided.");
+ return -EINVAL;
+ }
+ ret = request_firmware(&fw, se_img_file_to_load, priv->dev);
+ if (ret)
+ return ret;
+
+ if (fw->size > U32_MAX) {
+ ret = -EFBIG;
+ goto exit;
+ }
+ dev_info(priv->dev, "loading firmware %s.", se_img_file_to_load);
+
+ se_fw_buf_len = fw->size;
+ ret = get_shared_mem_slot(priv->priv_dev_ctx,
+ &se_fw_buf_len, &se_fw_dma_addr,
+ &se_fw_buf);
+ if (ret) {
+ dev_err(priv->dev, "Failed to allocate firmware shared buffer: %d\n",
+ ret);
+ goto exit;
+ }
+
+ memcpy(se_fw_buf, fw->data, fw->size);
+ ret = ele_fw_authenticate(priv, se_fw_dma_addr, se_fw_dma_addr);
+ if (ret < 0) {
+ dev_err(priv->dev,
+ "Error %pe: Authenticate & load SE firmware %s.",
+ ERR_PTR(ret), se_img_file_to_load);
+ ret = -EPERM;
+ }
+ if (!se_is_fw_busy_ctx(priv->priv_dev_ctx))
+ se_dev_ctx_shared_mem_cleanup(priv->priv_dev_ctx);
+exit:
+ release_firmware(fw);
+
+ return ret;
+}
+
+static int se_load_firmware(struct se_if_priv *priv)
+{
+ struct se_fw_load_info *load_fw = get_load_fw_instance(priv);
+ int ret = 0;
+
+ guard(mutex)(&load_fw->load_fw_lock);
+ if (!load_fw->is_fw_tobe_loaded)
+ return 0;
+
+ if (load_fw->imem.state == ELE_IMEM_STATE_BAD) {
+ ret = load_firmware(priv, load_fw->se_fw_img_nm->prim_fw_nm_in_rfs);
+ if (ret) {
+ dev_err(priv->dev, "Failed to load boot firmware.");
+ return -EPERM;
+ }
+ }
+
+ ret = load_firmware(priv, load_fw->se_fw_img_nm->seco_fw_nm_in_rfs);
+ if (ret) {
+ dev_err(priv->dev, "Failed to load runtime firmware.");
+ return -EPERM;
+ }
+
+ load_fw->is_fw_tobe_loaded = false;
+
+ return ret;
+}
+
+static int init_se_shared_mem(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt;
+ struct se_if_priv *priv = dev_ctx->priv;
+
+ INIT_LIST_HEAD(&se_shared_mem_mgmt->pending_out);
+ INIT_LIST_HEAD(&se_shared_mem_mgmt->pending_in);
+
+ /*
+ * Allocate some memory for data exchanges with S40x.
+ * This will be used for data not requiring secure memory.
+ */
+ se_shared_mem_mgmt->non_secure_mem.ptr =
+ dma_alloc_coherent(priv->dev, MAX_DATA_SIZE_PER_USER,
+ &se_shared_mem_mgmt->non_secure_mem.dma_addr,
+ GFP_KERNEL);
+ if (!se_shared_mem_mgmt->non_secure_mem.ptr)
+ return -ENOMEM;
+
+ se_shared_mem_mgmt->non_secure_mem.size = MAX_DATA_SIZE_PER_USER;
+ se_shared_mem_mgmt->non_secure_mem.pos = 0;
+
+ return 0;
+}
+
+static void cleanup_se_shared_mem(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt;
+ struct se_if_priv *priv = dev_ctx->priv;
+
+ /*
+ * Free any se_buf_desc items that were never consumed (e.g. when the
+ * fd is closed while pending I/O buffers are still listed). This must
+ * happen before the DMA backing memory is released to avoid a leak.
+ */
+ se_dev_ctx_shared_mem_cleanup(dev_ctx);
+
+ /* Guard against being called before shared memory was ever allocated
+ * (e.g. probe failure before dma_alloc_coherent succeeded).
+ */
+ if (!se_shared_mem_mgmt->non_secure_mem.ptr)
+ return;
+
+ /* Free non-secure shared buffer. */
+ dma_free_coherent(priv->dev, MAX_DATA_SIZE_PER_USER,
+ se_shared_mem_mgmt->non_secure_mem.ptr,
+ se_shared_mem_mgmt->non_secure_mem.dma_addr);
+
+ se_shared_mem_mgmt->non_secure_mem.ptr = NULL;
+ se_shared_mem_mgmt->non_secure_mem.dma_addr = 0;
+ se_shared_mem_mgmt->non_secure_mem.size = 0;
+ se_shared_mem_mgmt->non_secure_mem.pos = 0;
+}
+
+/* Need to copy the output data to user-device context.
+ */
+static int se_dev_ctx_cpy_out_data(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt;
+ struct se_if_priv *priv = dev_ctx->priv;
+ struct se_buf_desc *b_desc, *temp;
+ bool do_cpy = true;
+
+ list_for_each_entry_safe(b_desc, temp, &se_shared_mem_mgmt->pending_out, link) {
+ if (b_desc->usr_buf_ptr && b_desc->shared_buf_ptr && do_cpy) {
+ dev_dbg(priv->dev, "Copying output data to user.");
+ if (do_cpy && copy_to_user(b_desc->usr_buf_ptr,
+ b_desc->shared_buf_ptr,
+ b_desc->size)) {
+ dev_err(priv->dev, "Failure copying output data to user.");
+ do_cpy = false;
+ }
+ }
+
+ if (b_desc->shared_buf_ptr)
+ memset(b_desc->shared_buf_ptr, 0, b_desc->size);
+
+ list_del(&b_desc->link);
+ kfree(b_desc);
+ }
+
+ return do_cpy ? 0 : -EFAULT;
+}
+
+/*
+ * Clean the used Shared Memory space,
+ * whether its Input Data copied from user buffers, or
+ * Data received from FW.
+ */
+void se_dev_ctx_shared_mem_cleanup(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt;
+ struct list_head *pending_lists[] = {&se_shared_mem_mgmt->pending_in,
+ &se_shared_mem_mgmt->pending_out};
+ struct se_buf_desc *b_desc, *temp;
+ bool is_fw_busy_dev_ctx;
+ int i;
+
+ /*
+ * If this context is the one that caused a firmware timeout the shared
+ * DMA buffers may still be actively read/written by the firmware.
+ */
+ is_fw_busy_dev_ctx = se_is_fw_busy_ctx(dev_ctx);
+
+ for (i = 0; i < ARRAY_SIZE(pending_lists); i++) {
+ list_for_each_entry_safe(b_desc, temp, pending_lists[i], link) {
+ if (!is_fw_busy_dev_ctx && b_desc->shared_buf_ptr)
+ memset(b_desc->shared_buf_ptr, 0, b_desc->size);
+
+ list_del(&b_desc->link);
+ kfree(b_desc);
+ }
+ }
+ if (!is_fw_busy_dev_ctx)
+ se_shared_mem_mgmt->non_secure_mem.pos = 0;
+}
+
+static struct se_buf_desc *add_b_desc_to_pending_list(void *shared_ptr_with_pos,
+ struct se_ioctl_setup_iobuf *io,
+ struct se_if_device_ctx *dev_ctx)
+{
+ struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt;
+ struct se_buf_desc *b_desc = NULL;
+
+ b_desc = kzalloc_obj(*b_desc, GFP_KERNEL);
+ if (!b_desc)
+ return ERR_PTR(-ENOMEM);
+
+ b_desc->shared_buf_ptr = shared_ptr_with_pos;
+ b_desc->usr_buf_ptr = u64_to_user_ptr(io->user_buf);
+ b_desc->size = io->length;
+
+ if (io->flags & SE_IO_BUF_FLAGS_IS_INPUT) {
+ /*
+ * buffer is input:
+ * add an entry in the "pending input buffers" list so
+ * that copied data can be cleaned from shared memory
+ * later.
+ */
+ list_add_tail(&b_desc->link, &se_shared_mem_mgmt->pending_in);
+ } else {
+ /*
+ * buffer is output:
+ * add an entry in the "pending out buffers" list so data
+ * can be copied to user space when receiving Secure-Enclave
+ * response.
+ */
+ list_add_tail(&b_desc->link, &se_shared_mem_mgmt->pending_out);
+ }
+
+ return b_desc;
+}
+
+static void se_if_open_gate_release(struct kref *kref)
+{
+ struct se_if_open_gate *gate =
+ container_of(kref, struct se_if_open_gate, refcount);
+
+ kfree(gate);
+}
+
+static bool se_if_open_gate_get(struct se_if_open_gate *gate)
+{
+ if (!gate)
+ return false;
+
+ return kref_get_unless_zero(&gate->refcount);
+}
+
+static void se_if_open_gate_put(struct se_if_open_gate *gate)
+{
+ if (gate)
+ kref_put(&gate->refcount, se_if_open_gate_release);
+}
+
static int init_misc_device_context(struct se_if_priv *priv, int ch_id,
- struct se_if_device_ctx **new_dev_ctx)
+ struct se_if_device_ctx **new_dev_ctx,
+ const struct file_operations *se_if_fops)
{
const char *err_str = "Failed to allocate memory";
struct se_if_device_ctx *dev_ctx;
+ struct se_if_open_gate *gate = NULL;
int ret = -ENOMEM;
dev_ctx = kzalloc_obj(*dev_ctx, GFP_KERNEL);
@@ -227,18 +487,907 @@ static int init_misc_device_context(struct se_if_priv *priv, int ch_id,
if (!dev_ctx->devname)
goto exit;
+ mutex_init(&dev_ctx->fops_lock);
+
dev_ctx->priv = priv;
+ kref_init(&dev_ctx->refcount);
+ dev_ctx->cleanup_done = false;
*new_dev_ctx = dev_ctx;
+ set_se_rcv_msg_timeout(dev_ctx, SE_RCV_MSG_DEFAULT_TIMEOUT);
+ ret = init_se_shared_mem(dev_ctx);
+ if (ret < 0) {
+ kfree(dev_ctx->devname);
+ kfree(dev_ctx);
+ *new_dev_ctx = NULL;
+
+ return ret;
+ }
+
+ gate = kzalloc_obj(*gate, GFP_KERNEL);
+ if (!gate)
+ goto exit;
+
+ mutex_init(&gate->lock);
+ kref_init(&gate->refcount); /* device-owned reference */
+ gate->priv = priv;
+ gate->dying = false;
+ priv->open_gate = gate;
+
+ /*
+ * The miscdevice storage is now owned by the open gate object.
+ * priv->priv_dev_ctx still keeps a pointer to that miscdevice.
+ */
+ dev_ctx->miscdev = &gate->miscdev;
+
+ dev_ctx->miscdev->name = dev_ctx->devname;
+ dev_ctx->miscdev->minor = MISC_DYNAMIC_MINOR;
+ dev_ctx->miscdev->fops = se_if_fops;
+ dev_ctx->miscdev->parent = priv->dev;
+ ret = misc_register(dev_ctx->miscdev);
+ if (ret) {
+ err_str = "Failed to register misc device.";
+ goto exit;
+ }
return 0;
exit:
*new_dev_ctx = NULL;
-
+ if (gate) {
+ priv->open_gate = NULL;
+ se_if_open_gate_put(gate);
+ }
kfree(dev_ctx->devname);
kfree(dev_ctx);
return dev_err_probe(priv->dev, ret, "%s", err_str);
}
+static void se_if_priv_release(struct kref *kref)
+{
+ struct se_if_priv *priv = container_of(kref, struct se_if_priv, refcount);
+
+ /* Free priv_dev_ctx if it exists */
+ if (priv->priv_dev_ctx) {
+ /*
+ * miscdev storage belongs to open_gate, not directly to
+ * priv_dev_ctx. The gate should already have been detached
+ * from priv during teardown.
+ */
+
+ kfree(priv->priv_dev_ctx->devname);
+ kfree(priv->priv_dev_ctx);
+ priv->priv_dev_ctx = NULL;
+ }
+
+ /*
+ * Be defensive: if teardown did not already drop the device-owned
+ * gate reference for some reason, release it here.
+ */
+ if (priv->open_gate) {
+ se_if_open_gate_put(priv->open_gate);
+ priv->open_gate = NULL;
+ }
+
+ /* Free any remaining resources that weren't devm-managed */
+ kfree(priv);
+}
+
+static void se_if_dev_ctx_release(struct kref *kref)
+{
+ struct se_if_device_ctx *dev_ctx =
+ container_of(kref, struct se_if_device_ctx, refcount);
+ struct se_if_priv *priv = dev_ctx->priv;
+
+ kfree(dev_ctx);
+
+ /* drop the priv reference owned by this device context */
+ kref_put(&priv->refcount, se_if_priv_release);
+}
+
+static void se_clear_fw_busy(struct se_if_priv *priv)
+{
+ struct se_if_device_ctx *dev_ctx = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->fw_busy_lock, flags);
+ dev_ctx = priv->fw_busy_dev_ctx;
+ priv->fw_busy_dev_ctx = NULL;
+ atomic_set(&priv->fw_busy, 0);
+ spin_unlock_irqrestore(&priv->fw_busy_lock, flags);
+
+ if (!dev_ctx)
+ return;
+
+ scoped_guard(mutex, &dev_ctx->fops_lock) {
+ if (dev_ctx->cleanup_done)
+ cleanup_se_shared_mem(dev_ctx);
+ }
+
+ kref_put(&dev_ctx->refcount, se_if_dev_ctx_release);
+}
+
+void unset_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+ struct se_api_msg *old_rx_msg = NULL;
+ struct se_clbk_handle *se_clbk_hdl;
+ unsigned long flags;
+
+ lockdep_assert_held(&priv->modify_lock);
+ se_clbk_hdl = &priv->cmd_receiver_clbk_hdl;
+
+ if (se_clbk_hdl->dev_ctx == dev_ctx) {
+ spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
+ old_rx_msg = se_clbk_hdl->rx_msg;
+ se_clbk_hdl->dev_ctx = NULL;
+ se_clbk_hdl->rx_msg = NULL;
+ se_clbk_hdl->rx_msg_sz = 0;
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+
+ kfree(old_rx_msg);
+ complete_all(&se_clbk_hdl->done);
+ }
+}
+
+int set_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+ struct se_api_msg *new_rx_msg = NULL;
+ struct se_clbk_handle *se_clbk_hdl;
+ unsigned long flags;
+
+ se_clbk_hdl = &priv->cmd_receiver_clbk_hdl;
+ guard(mutex)(&priv->modify_lock);
+ if (se_clbk_hdl->dev_ctx == dev_ctx)
+ return 0;
+
+ if (se_clbk_hdl->dev_ctx)
+ return -EBUSY;
+
+ if (!se_clbk_hdl->rx_msg) {
+ new_rx_msg = kzalloc(MAX_NVM_MSG_LEN, GFP_KERNEL);
+ if (!new_rx_msg)
+ return -ENOMEM;
+ }
+ spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
+ if (new_rx_msg)
+ se_clbk_hdl->rx_msg = new_rx_msg;
+ reinit_completion(&se_clbk_hdl->done);
+ se_clbk_hdl->rx_msg_sz = MAX_NVM_MSG_LEN;
+ se_clbk_hdl->dev_ctx = dev_ctx;
+ dev_ctx->rcv_msg_timeout_jiffies = MAX_SCHEDULE_TIMEOUT;
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+
+ return 0;
+}
+
+static void dlink_dev_ctx(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+
+ unset_dev_ctx_as_command_receiver(dev_ctx);
+
+ if (!list_empty(&dev_ctx->link)) {
+ list_del_init(&dev_ctx->link);
+ priv->active_devctx_count--;
+ }
+}
+
+bool se_is_fw_busy_ctx(struct se_if_device_ctx *dev_ctx)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+ unsigned long flags;
+ bool match;
+
+ spin_lock_irqsave(&priv->fw_busy_lock, flags);
+ match = priv->fw_busy_dev_ctx == dev_ctx;
+ spin_unlock_irqrestore(&priv->fw_busy_lock, flags);
+
+ return match;
+}
+
+static void cleanup_dev_ctx(struct se_if_device_ctx *dev_ctx, bool is_fclose)
+{
+ bool already_done;
+
+ scoped_guard(mutex, &dev_ctx->fops_lock) {
+ already_done = dev_ctx->cleanup_done;
+ if (!already_done) {
+ /*
+ * fw_busy is caused by one timed-out synchronous transaction.
+ * Only that transaction's dev_ctx may still have coherent
+ * memory referenced by FW. Do not skip cleanup for unrelated
+ * contexts while fw_busy is set.
+ */
+ if (se_is_fw_busy_ctx(dev_ctx))
+ dev_warn(dev_ctx->priv->dev,
+ "%s: deferring shared memory cleanup while FW is busy\n",
+ dev_ctx->devname);
+ else
+ cleanup_se_shared_mem(dev_ctx);
+
+ kfree(dev_ctx->devname);
+ dev_ctx->devname = NULL;
+ dev_ctx->cleanup_done = true;
+ }
+ }
+
+ if (is_fclose)
+ kref_put(&dev_ctx->refcount, se_if_dev_ctx_release);
+}
+
+static void dlink_n_cleanup_dev_ctx(struct se_if_device_ctx *dev_ctx, bool is_fclose)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+
+ if (is_fclose) {
+ scoped_guard(mutex, &priv->modify_lock)
+ dlink_dev_ctx(dev_ctx);
+ }
+
+ cleanup_dev_ctx(dev_ctx, is_fclose);
+}
+
+static int init_device_context(struct se_if_priv *priv, int ch_id,
+ struct se_if_device_ctx **new_dev_ctx)
+{
+ struct se_if_device_ctx *dev_ctx;
+ int ret = 0;
+
+ dev_ctx = kzalloc_obj(*dev_ctx, GFP_KERNEL);
+
+ if (!dev_ctx)
+ return -ENOMEM;
+
+ dev_ctx->devname = kasprintf(GFP_KERNEL, "%s0_ch%d",
+ get_se_if_name(priv->if_defs->se_if_type),
+ ch_id);
+ if (!dev_ctx->devname) {
+ kfree(dev_ctx);
+ return -ENOMEM;
+ }
+
+ mutex_init(&dev_ctx->fops_lock);
+ kref_init(&dev_ctx->refcount);
+ dev_ctx->priv = priv;
+ dev_ctx->cleanup_done = false;
+ INIT_LIST_HEAD(&dev_ctx->link);
+ set_se_rcv_msg_timeout(dev_ctx, SE_RCV_MSG_LONG_TIMEOUT);
+ *new_dev_ctx = dev_ctx;
+
+ ret = init_se_shared_mem(dev_ctx);
+ if (ret < 0) {
+ kfree(dev_ctx->devname);
+ kfree(dev_ctx);
+ *new_dev_ctx = NULL;
+
+ return ret;
+ }
+
+ /* Take a reference to priv for this device context */
+ kref_get(&priv->refcount);
+
+ scoped_guard(mutex, &priv->modify_lock) {
+ list_add_tail(&dev_ctx->link, &priv->dev_ctx_list);
+ priv->active_devctx_count++;
+ }
+
+ return ret;
+}
+
+static int se_ioctl_cmd_snd_rcv_cleanup(struct se_if_device_ctx *dev_ctx, void __user *uarg,
+ struct se_ioctl_cmd_snd_rcv_rsp_info *cmd_snd_rcv_rsp_info)
+{
+ /* shared memory is allocated before this IOCTL */
+ se_dev_ctx_shared_mem_cleanup(dev_ctx);
+
+ if (cmd_snd_rcv_rsp_info->rx_buf_sz &&
+ copy_to_user(uarg, cmd_snd_rcv_rsp_info, sizeof(*cmd_snd_rcv_rsp_info))) {
+ dev_err(dev_ctx->priv->dev, "%s: Failed to copy cmd_snd_rcv_rsp_info to user.",
+ dev_ctx->devname);
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
+static int se_ioctl_cmd_snd_rcv_rsp_handler(struct se_if_device_ctx *dev_ctx,
+ void __user *uarg)
+{
+ struct se_ioctl_cmd_snd_rcv_rsp_info cmd_snd_rcv_rsp_info = {0};
+ struct se_if_priv *priv = dev_ctx->priv;
+ int rsp_status_err = 0;
+ int cleanup_err = 0;
+ int err = 0;
+
+ if (copy_from_user(&cmd_snd_rcv_rsp_info, uarg,
+ sizeof(cmd_snd_rcv_rsp_info))) {
+ dev_err(priv->dev,
+ "%s: Failed to copy cmd_snd_rcv_rsp_info from user.",
+ dev_ctx->devname);
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return -EFAULT;
+ }
+
+ if (cmd_snd_rcv_rsp_info.tx_buf_sz < SE_MU_HDR_SZ ||
+ cmd_snd_rcv_rsp_info.tx_buf_sz > MAX_ALLOWED_TX_MSG_SZ) {
+ dev_err(priv->dev, "%s: User buffer too small/large(%d < %d)",
+ dev_ctx->devname, cmd_snd_rcv_rsp_info.tx_buf_sz,
+ cmd_snd_rcv_rsp_info.tx_buf_sz < SE_MU_HDR_SZ ? SE_MU_HDR_SZ :
+ MAX_ALLOWED_TX_MSG_SZ);
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return -ENOSPC;
+ }
+
+ struct se_api_msg *tx_msg __free(kfree) =
+ memdup_user(u64_to_user_ptr(cmd_snd_rcv_rsp_info.tx_buf),
+ cmd_snd_rcv_rsp_info.tx_buf_sz);
+ if (IS_ERR(tx_msg)) {
+ err = PTR_ERR(tx_msg);
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return err;
+ }
+
+ err = se_chk_tx_msg_hdr(dev_ctx, &tx_msg->header);
+ if (err) {
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return err;
+ }
+
+ if (cmd_snd_rcv_rsp_info.rx_buf_sz < SE_MU_HDR_SZ ||
+ cmd_snd_rcv_rsp_info.rx_buf_sz > MAX_ALLOWED_RX_MSG_SZ) {
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return -EINVAL;
+ }
+
+ if (tx_msg->header.tag != priv->if_defs->cmd_tag) {
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return -EINVAL;
+ }
+
+ if (tx_msg->header.ver == priv->if_defs->fw_api_ver &&
+ get_load_fw_instance(priv)->is_fw_tobe_loaded) {
+ err = se_load_firmware(priv);
+ if (err) {
+ dev_err(priv->dev, "Could not send msg as FW is not loaded.");
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return -EPERM;
+ }
+ }
+
+ struct se_api_msg *rx_msg __free(kfree) =
+ kzalloc(cmd_snd_rcv_rsp_info.rx_buf_sz, GFP_KERNEL);
+ if (!rx_msg) {
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return -ENOMEM;
+ }
+
+ err = ele_msg_send_rcv(dev_ctx, tx_msg, cmd_snd_rcv_rsp_info.tx_buf_sz,
+ rx_msg, cmd_snd_rcv_rsp_info.rx_buf_sz);
+ if (err < 0) {
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+
+ return err;
+ }
+
+ dev_dbg(priv->dev, "%s: %s %s.", dev_ctx->devname, __func__,
+ "message received, start transmit to user");
+
+ rsp_status_err =
+ se_val_rsp_hdr_n_status(priv, rx_msg, tx_msg->header.command,
+ cmd_snd_rcv_rsp_info.rx_buf_sz,
+ tx_msg->header.ver == priv->if_defs->base_api_ver);
+
+ if (!rsp_status_err) {
+ err = se_dev_ctx_cpy_out_data(dev_ctx);
+ if (err < 0) {
+ se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+ return err;
+ }
+ }
+
+ /* Copy data from the buffer */
+ print_hex_dump_debug("to user ", DUMP_PREFIX_OFFSET, 4, 4, rx_msg,
+ cmd_snd_rcv_rsp_info.rx_buf_sz, false);
+
+ if (copy_to_user(u64_to_user_ptr(cmd_snd_rcv_rsp_info.rx_buf), rx_msg,
+ cmd_snd_rcv_rsp_info.rx_buf_sz)) {
+ dev_err(priv->dev, "%s: Failed to copy to user.", dev_ctx->devname);
+ err = -EFAULT;
+ }
+
+ cleanup_err = se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info);
+
+ if (cleanup_err && !err)
+ err = cleanup_err;
+
+ if (!err && !rsp_status_err)
+ fw_api_specific_ops(dev_ctx, &tx_msg->header);
+
+ return err;
+}
+
+static int se_ioctl_get_mu_info(struct se_if_device_ctx *dev_ctx,
+ void __user *uarg)
+{
+ struct se_if_priv *priv = dev_ctx->priv;
+ struct se_ioctl_get_if_info if_info;
+ struct se_if_node *if_node;
+ int err = 0;
+
+ if_node = container_of(priv->if_defs, typeof(*if_node), if_defs);
+
+ if_info.se_if_id = 0;
+ if_info.interrupt_idx = 0;
+ if_info.tz = 0;
+ if_info.did = 0;
+ if_info.cmd_tag = priv->if_defs->cmd_tag;
+ if_info.rsp_tag = priv->if_defs->rsp_tag;
+ if_info.success_tag = priv->if_defs->success_tag;
+ if_info.base_api_ver = priv->if_defs->base_api_ver;
+ if_info.fw_api_ver = priv->if_defs->fw_api_ver;
+
+ dev_dbg(priv->dev, "%s: info [se_if_id: %d, irq_idx: %d, tz: 0x%x, did: 0x%x].",
+ dev_ctx->devname, if_info.se_if_id, if_info.interrupt_idx, if_info.tz,
+ if_info.did);
+
+ if (copy_to_user(uarg, &if_info, sizeof(if_info))) {
+ dev_err(priv->dev, "%s: Failed to copy mu info to user.",
+ dev_ctx->devname);
+ err = -EFAULT;
+ }
+
+ return err;
+}
+
+static void rollback_shared_mem_pos(struct se_if_device_ctx *dev_ctx, u32 length)
+{
+ struct se_shared_mem *shared_mem = NULL;
+
+ shared_mem = &dev_ctx->se_shared_mem_mgmt.non_secure_mem;
+
+ if (WARN_ON_ONCE(length > shared_mem->pos)) {
+ shared_mem->pos = 0;
+ return;
+ }
+
+ shared_mem->pos -= length;
+}
+
+int get_shared_mem_slot(struct se_if_device_ctx *dev_ctx,
+ u32 *length, dma_addr_t *ele_dma_addr, void **ptr)
+{
+ struct se_shared_mem *shared_mem = NULL;
+ bool is_fw_busy_dev_ctx;
+ size_t aligned_len = 0;
+ u32 pos;
+
+ /*
+ * If this context is the one that caused a firmware timeout the shared
+ * DMA buffers may still be actively read/written by the firmware.
+ */
+ is_fw_busy_dev_ctx = se_is_fw_busy_ctx(dev_ctx);
+ if (is_fw_busy_dev_ctx)
+ return -EBUSY;
+
+ aligned_len = round_up((size_t)*length, 8);
+ if (aligned_len < *length) {
+ dev_err(dev_ctx->priv->dev, "%s: Invalid buffer length.",
+ dev_ctx->devname);
+ return -EINVAL;
+ }
+
+ /* No specific requirement for this buffer. */
+ shared_mem = &dev_ctx->se_shared_mem_mgmt.non_secure_mem;
+
+ /* Check there is enough space in the shared memory. */
+ dev_dbg(dev_ctx->priv->dev, "%s: req_size = %zd, max_size= %d, curr_pos = %d",
+ dev_ctx->devname, aligned_len, shared_mem->size,
+ shared_mem->pos);
+
+ if (shared_mem->size < shared_mem->pos ||
+ aligned_len > (shared_mem->size - shared_mem->pos)) {
+ dev_err(dev_ctx->priv->dev, "%s: Not enough space in shared memory.",
+ dev_ctx->devname);
+ return -ENOMEM;
+ }
+
+ /* Allocate space in shared memory. 8 bytes aligned. */
+ pos = shared_mem->pos;
+ shared_mem->pos += aligned_len;
+ *ele_dma_addr = (u64)shared_mem->dma_addr + pos;
+ *ptr = shared_mem->ptr + pos;
+ *length = aligned_len;
+
+ memset(shared_mem->ptr + pos, 0, aligned_len);
+
+ return 0;
+}
+
+/*
+ * Copy a buffer of data to/from the user and return the address to use in
+ * messages
+ */
+static int se_ioctl_setup_iobuf_handler(struct se_if_device_ctx *dev_ctx,
+ void __user *uarg)
+{
+ struct se_ioctl_setup_iobuf io = {0};
+ struct se_buf_desc *b_desc = NULL;
+ void *dma_buf_ptr = NULL;
+ dma_addr_t ele_dma_addr;
+ u32 aligned_len = 0;
+ int err = 0;
+
+ if (copy_from_user(&io, uarg, sizeof(io))) {
+ dev_err(dev_ctx->priv->dev, "%s: Failed copy iobuf config from user.",
+ dev_ctx->devname);
+ return -EFAULT;
+ }
+
+ dev_dbg(dev_ctx->priv->dev, "%s: io [buf: %p(%d) flag: %x].", dev_ctx->devname,
+ u64_to_user_ptr(io.user_buf), io.length, io.flags);
+
+ if (io.length == 0 || !io.user_buf) {
+ /*
+ * Accept NULL pointers since some buffers are optional
+ * in FW commands. In this case we should return 0 as
+ * pointer to be embedded into the message.
+ * Skip all data copy part of code below.
+ */
+ io.ele_addr = 0;
+ goto copy;
+ }
+
+ aligned_len = io.length;
+ err = get_shared_mem_slot(dev_ctx, &aligned_len, &ele_dma_addr, &dma_buf_ptr);
+ if (err)
+ return err;
+
+ io.ele_addr = ele_dma_addr;
+ if ((io.flags & SE_IO_BUF_FLAGS_IS_INPUT) ||
+ (io.flags & SE_IO_BUF_FLAGS_IS_IN_OUT)) {
+ /*
+ * buffer is input:
+ * copy data from user space to this allocated buffer.
+ */
+ if (copy_from_user(dma_buf_ptr, u64_to_user_ptr(io.user_buf),
+ io.length)) {
+ dev_err(dev_ctx->priv->dev,
+ "%s: Failed copy data to shared memory.",
+ dev_ctx->devname);
+ err = -EFAULT;
+ goto rollback;
+ }
+ }
+
+ b_desc = add_b_desc_to_pending_list(dma_buf_ptr, &io, dev_ctx);
+ if (IS_ERR(b_desc)) {
+ err = PTR_ERR(b_desc);
+ dev_err(dev_ctx->priv->dev, "%s: Failed to allocate/link b_desc.",
+ dev_ctx->devname);
+ goto rollback;
+ }
+
+copy:
+ /* Provide the EdgeLock Enclave address to user space only if success.*/
+ if (copy_to_user(uarg, &io, sizeof(io))) {
+ dev_err(dev_ctx->priv->dev, "%s: Failed to copy iobuff setup to user.",
+ dev_ctx->devname);
+ err = -EFAULT;
+ goto rollback;
+ }
+ return err;
+
+rollback:
+ if (!IS_ERR_OR_NULL(b_desc)) {
+ list_del(&b_desc->link);
+ kfree(b_desc);
+ }
+
+ if (dma_buf_ptr && aligned_len) {
+ memset(dma_buf_ptr, 0, aligned_len);
+ rollback_shared_mem_pos(dev_ctx, aligned_len);
+ }
+
+ return err;
+}
+
+/* IOCTL to provide SoC information */
+static int se_ioctl_get_se_soc_info_handler(struct se_if_device_ctx *dev_ctx,
+ void __user *uarg)
+{
+ struct se_ioctl_get_soc_info soc_info;
+ int err = -EINVAL;
+
+ soc_info.soc_id = get_se_soc_id(dev_ctx->priv);
+ soc_info.soc_rev = var_se_info.soc_rev;
+
+ err = copy_to_user(uarg, (u8 *)(&soc_info), sizeof(soc_info));
+ if (err) {
+ dev_err(dev_ctx->priv->dev, "%s: Failed to copy soc info to user.",
+ dev_ctx->devname);
+ err = -EFAULT;
+ }
+
+ return err;
+}
+
+/*
+ * File operations for user-space
+ */
+
+/* Write a message to the MU. */
+static ssize_t se_if_fops_write(struct file *fp, const char __user *buf,
+ size_t size, loff_t *ppos)
+{
+ struct se_if_device_ctx *dev_ctx = fp->private_data;
+ struct se_if_priv *priv;
+ int err;
+
+ scoped_cond_guard(mutex_intr, return -EBUSY, &dev_ctx->fops_lock) {
+ if (dev_ctx->cleanup_done)
+ return -ENODEV;
+
+ priv = dev_ctx->priv;
+
+ dev_dbg(priv->dev, "%s: write from buf (%p)%zu, ppos=%lld.", dev_ctx->devname,
+ buf, size, ((ppos) ? *ppos : 0));
+
+ if (dev_ctx != priv->cmd_receiver_clbk_hdl.dev_ctx)
+ return -EINVAL;
+
+ if (size < SE_MU_HDR_SZ || size > MAX_ALLOWED_TX_MSG_SZ) {
+ dev_err(priv->dev, "%s: User buffer too small/large(%zu < %d)",
+ dev_ctx->devname, size,
+ size < SE_MU_HDR_SZ ? SE_MU_HDR_SZ :
+ MAX_ALLOWED_TX_MSG_SZ);
+ return -ENOSPC;
+ }
+
+ struct se_api_msg *tx_msg __free(kfree) = memdup_user(buf, size);
+ if (IS_ERR(tx_msg))
+ return PTR_ERR(tx_msg);
+
+ err = se_chk_tx_msg_hdr(dev_ctx, &tx_msg->header);
+ if (err)
+ return err;
+
+ print_hex_dump_debug("from user ", DUMP_PREFIX_OFFSET, 4, 4,
+ tx_msg, size, false);
+
+ err = ele_msg_send(dev_ctx, tx_msg, size);
+
+ return err;
+ }
+}
+
+/*
+ * Read a message from the MU.
+ * Blocking until a message is available.
+ */
+static ssize_t se_if_fops_read(struct file *fp, char __user *buf, size_t size,
+ loff_t *ppos)
+{
+ struct se_if_device_ctx *dev_ctx = fp->private_data;
+ struct se_if_priv *priv;
+ unsigned long flags;
+ size_t copy_len;
+ int err;
+
+ scoped_cond_guard(mutex_intr, return -EBUSY, &dev_ctx->fops_lock) {
+ if (dev_ctx->cleanup_done) {
+ priv->cmd_receiver_clbk_hdl.rx_msg_sz = 0;
+ se_dev_ctx_shared_mem_cleanup(dev_ctx);
+ return -ENODEV;
+ }
+
+ priv = dev_ctx->priv;
+
+ dev_dbg(priv->dev, "%s: read to buf %p(%zu), ppos=%lld.", dev_ctx->devname,
+ buf, size, ((ppos) ? *ppos : 0));
+
+ if (dev_ctx != priv->cmd_receiver_clbk_hdl.dev_ctx) {
+ priv->cmd_receiver_clbk_hdl.rx_msg_sz = 0;
+ se_dev_ctx_shared_mem_cleanup(dev_ctx);
+ return -EINVAL;
+ }
+ }
+
+ err = ele_msg_rcv(dev_ctx, &priv->cmd_receiver_clbk_hdl);
+ if (err < 0) {
+ if (err != -ERESTARTSYS)
+ dev_err(priv->dev,
+ "%s: Er[0x%x]: Signal Interrupted. Current act-dev-ctx count: %d.",
+ dev_ctx->devname, err, dev_ctx->priv->active_devctx_count);
+ goto exit;
+ }
+
+ /*
+ * Reacquire fops_lock before touching any dev_ctx state (pending lists,
+ * rx_msg) after the blocking wait. fops_lock was dropped before calling
+ * ele_msg_rcv(). If cleanup_dev_ctx() ran concurrently it could have
+ * freed the DMA buffers and the pending lists, leading to UAF and list
+ * corruption. Re-checking cleanup_done under fops_lock prevents that.
+ */
+ if (mutex_lock_interruptible(&dev_ctx->fops_lock)) {
+ err = -ERESTARTSYS;
+ goto exit;
+ }
+
+ if (dev_ctx->cleanup_done) {
+ err = -ENODEV;
+ goto exit_mutex_unlock;
+ }
+
+ spin_lock_irqsave(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags);
+ if (priv->cmd_receiver_clbk_hdl.dev_ctx != dev_ctx ||
+ !priv->cmd_receiver_clbk_hdl.rx_msg ||
+ !priv->cmd_receiver_clbk_hdl.rx_msg_sz) {
+ spin_unlock_irqrestore(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags);
+ err = -ENODEV;
+ goto exit_mutex_unlock;
+ }
+ spin_unlock_irqrestore(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags);
+
+ /* We may need to copy the output data to user before
+ * delivering the completion message.
+ */
+ err = se_dev_ctx_cpy_out_data(dev_ctx);
+ if (err < 0)
+ goto exit_mutex_unlock;
+
+ /* Copy data from the buffer */
+ print_hex_dump_debug("to user ", DUMP_PREFIX_OFFSET, 4, 4,
+ priv->cmd_receiver_clbk_hdl.rx_msg,
+ priv->cmd_receiver_clbk_hdl.rx_msg_sz,
+ false);
+
+ copy_len = min(size, priv->cmd_receiver_clbk_hdl.rx_msg_sz);
+
+ if (copy_to_user(buf, priv->cmd_receiver_clbk_hdl.rx_msg, copy_len))
+ err = -EFAULT;
+ else
+ err = copy_len;
+
+exit_mutex_unlock:
+ mutex_unlock(&dev_ctx->fops_lock);
+
+exit:
+ spin_lock_irqsave(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags);
+ priv->cmd_receiver_clbk_hdl.rx_msg_sz = 0;
+ spin_unlock_irqrestore(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags);
+ se_dev_ctx_shared_mem_cleanup(dev_ctx);
+
+ return err;
+}
+
+/* Open a character device. */
+static int se_if_fops_open(struct inode *nd, struct file *fp)
+{
+ struct miscdevice *miscdev = fp->private_data;
+ struct se_if_open_gate *gate;
+ struct se_if_device_ctx *misc_dev_ctx;
+ struct se_if_device_ctx *dev_ctx;
+ struct se_if_priv *priv;
+ int err = 0;
+
+ gate = container_of(miscdev, struct se_if_open_gate, miscdev);
+
+ if (!se_if_open_gate_get(gate))
+ return -ENODEV;
+
+ if (mutex_lock_interruptible(&gate->lock)) {
+ se_if_open_gate_put(gate);
+ return -EBUSY;
+ }
+
+ if (gate->dying || !gate->priv ||
+ !kref_get_unless_zero(&gate->priv->refcount)) {
+ err = -ENODEV;
+ goto out_unlock_gate;
+ }
+
+ priv = gate->priv;
+ mutex_unlock(&gate->lock);
+
+ misc_dev_ctx = priv->priv_dev_ctx;
+
+ if (mutex_lock_interruptible(&misc_dev_ctx->fops_lock)) {
+ err = -EBUSY;
+ goto out_put_priv;
+ }
+
+ if (misc_dev_ctx->cleanup_done) {
+ err = -ENODEV;
+ goto out_unlock_misc;
+ }
+
+ priv->dev_ctx_mono_count++;
+ err = init_device_context(priv, priv->dev_ctx_mono_count, &dev_ctx);
+ if (err) {
+ dev_err(priv->dev, "Failed[0x%x] to create dev-ctx.", err);
+ goto out_unlock_misc;
+ }
+
+ fp->private_data = dev_ctx;
+
+out_unlock_misc:
+ mutex_unlock(&misc_dev_ctx->fops_lock);
+out_put_priv:
+ kref_put(&priv->refcount, se_if_priv_release);
+ se_if_open_gate_put(gate);
+ return err;
+out_unlock_gate:
+ mutex_unlock(&gate->lock);
+ se_if_open_gate_put(gate);
+ return err;
+}
+
+/* Close a character device. */
+static int se_if_fops_close(struct inode *nd, struct file *fp)
+{
+ struct se_if_device_ctx *dev_ctx = fp->private_data;
+
+ dlink_n_cleanup_dev_ctx(dev_ctx, true);
+
+ return 0;
+}
+
+/* IOCTL entry point of a character device */
+static long se_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
+{
+ struct se_if_device_ctx *dev_ctx = fp->private_data;
+ struct se_if_priv *priv;
+ void __user *uarg = (void __user *)arg;
+ long err;
+
+ /* Prevent race during change of device context */
+ scoped_cond_guard(mutex_intr, return -EBUSY, &dev_ctx->fops_lock) {
+ if (dev_ctx->cleanup_done)
+ return -ENODEV;
+
+ priv = dev_ctx->priv;
+
+ switch (cmd) {
+ case SE_IOCTL_ENABLE_CMD_RCV: {
+ err = set_dev_ctx_as_command_receiver(dev_ctx);
+ if (err)
+ dev_err(priv->dev, "Failed to register %s as CMD-Receiver: %ld\n",
+ dev_ctx->devname, err);
+ break;
+ }
+ case SE_IOCTL_GET_MU_INFO:
+ err = se_ioctl_get_mu_info(dev_ctx, uarg);
+ break;
+ case SE_IOCTL_SETUP_IOBUF:
+ err = se_ioctl_setup_iobuf_handler(dev_ctx, uarg);
+ break;
+ case SE_IOCTL_GET_SOC_INFO:
+ err = se_ioctl_get_se_soc_info_handler(dev_ctx, uarg);
+ break;
+ case SE_IOCTL_CMD_SEND_RCV_RSP:
+ err = se_ioctl_cmd_snd_rcv_rsp_handler(dev_ctx, uarg);
+ break;
+ default:
+ err = -ENOTTY;
+ dev_dbg(priv->dev, "%s: IOCTL %.8x not supported.",
+ dev_ctx->devname, cmd);
+ }
+ }
+
+ return err;
+}
+
+/* Char driver setup */
+static const struct file_operations se_if_fops = {
+ .open = se_if_fops_open,
+ .owner = THIS_MODULE,
+ .release = se_if_fops_close,
+ .unlocked_ioctl = se_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
+ .read = se_if_fops_read,
+ .write = se_if_fops_write,
+};
+
/* interface for managed res to free a mailbox channel */
static void if_mbox_free_channel(void *mbox_chan)
{
@@ -269,6 +1418,7 @@ static int se_if_request_channel(struct device *dev, struct mbox_chan **chan,
static void se_if_probe_cleanup(void *plat_dev)
{
struct platform_device *pdev = plat_dev;
+ struct se_if_device_ctx *dev_ctx;
struct device *dev = &pdev->dev;
struct se_if_priv *priv;
@@ -276,6 +1426,62 @@ static void se_if_probe_cleanup(void *plat_dev)
if (!priv)
return;
+ /*
+ * Mark the private device context as cleanup_done first.
+ * This prevents new device contexts from being created in open().
+ */
+ if (priv->priv_dev_ctx) {
+ scoped_guard(mutex, &priv->modify_lock)
+ priv->priv_dev_ctx->cleanup_done = true;
+
+ if (priv->open_gate) {
+ scoped_guard(mutex, &priv->open_gate->lock) {
+ priv->open_gate->dying = true;
+ priv->open_gate->priv = NULL;
+ }
+ }
+
+ if (priv->priv_dev_ctx->miscdev)
+ misc_deregister(priv->priv_dev_ctx->miscdev);
+ }
+
+ while (true) {
+ dev_ctx = NULL;
+
+ scoped_guard(mutex, &priv->modify_lock) {
+ if (list_empty(&priv->dev_ctx_list))
+ goto out_done;
+
+ dev_ctx = list_first_entry(&priv->dev_ctx_list,
+ struct se_if_device_ctx, link);
+
+ /* pin this context so close() cannot free it under us */
+ kref_get(&dev_ctx->refcount);
+ dlink_dev_ctx(dev_ctx);
+ }
+
+ /*
+ * Local cleanup outside the global lock avoids ABBA deadlock
+ * with paths that already take dev_ctx->fops_lock first.
+ */
+ cleanup_dev_ctx(dev_ctx, false);
+ kref_put(&dev_ctx->refcount, se_if_dev_ctx_release);
+ }
+out_done:
+
+ /*
+ * A timed-out synchronous command may have retained a dev_ctx through
+ * priv->fw_busy_dev_ctx even after the fd was closed and the context was
+ * removed from dev_ctx_list. If no late response arrived, release that
+ * retained context during driver teardown.
+ *
+ * se_clear_fw_busy() is idempotent and internally checks
+ * priv->fw_busy_dev_ctx under fw_busy_lock.
+ */
+ complete_all(&priv->waiting_rsp_clbk_hdl.done);
+ se_clear_fw_busy(priv);
+ cancel_work_sync(&priv->fw_busy_work);
+
/*
* In se_if_request_channel(), passed the clean-up functional
* pointer reference as action to devm_add_action_or_reset().
@@ -296,12 +1502,16 @@ static void se_if_probe_cleanup(void *plat_dev)
dev_set_drvdata(dev, NULL);
- if (priv->priv_dev_ctx) {
- kfree(priv->priv_dev_ctx->devname);
- kfree(priv->priv_dev_ctx);
- priv->priv_dev_ctx = NULL;
- }
- kfree(priv);
+ /* Drop the initial reference - priv will be freed when last fd closes */
+ kref_put(&priv->refcount, se_if_priv_release);
+}
+
+static void se_fw_busy_work(struct work_struct *work)
+{
+ struct se_if_priv *priv =
+ container_of(work, struct se_if_priv, fw_busy_work);
+
+ se_clear_fw_busy(priv);
}
static int se_if_probe(struct platform_device *pdev)
@@ -324,15 +1534,22 @@ static int se_if_probe(struct platform_device *pdev)
return -ENOMEM;
priv->dev = dev;
+ kref_init(&priv->refcount);
priv->if_defs = &if_node->if_defs;
dev_set_drvdata(dev, priv);
mutex_init(&priv->se_if_cmd_lock);
+ mutex_init(&priv->modify_lock);
spin_lock_init(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock);
spin_lock_init(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock);
atomic_set(&priv->fw_busy, 0);
+ spin_lock_init(&priv->fw_busy_lock);
+ priv->fw_busy_dev_ctx = NULL;
+ INIT_WORK(&priv->fw_busy_work, se_fw_busy_work);
+
init_completion(&priv->waiting_rsp_clbk_hdl.done);
init_completion(&priv->cmd_receiver_clbk_hdl.done);
+ INIT_LIST_HEAD(&priv->dev_ctx_list);
ret = devm_add_action_or_reset(dev, se_if_probe_cleanup, pdev);
if (ret)
@@ -371,7 +1588,7 @@ static int se_if_probe(struct platform_device *pdev)
if (ret)
return dev_err_probe(dev, ret, "Failed to set 32-bit coherent DMA mask.");
- ret = init_misc_device_context(priv, 0, &priv->priv_dev_ctx);
+ ret = init_misc_device_context(priv, 0, &priv->priv_dev_ctx, &se_if_fops);
if (ret)
return dev_err_probe(dev, ret,
"Failed[0x%x] to create device contexts.",
diff --git a/drivers/firmware/imx/se_ctrl.h b/drivers/firmware/imx/se_ctrl.h
index 355d38684b1b..7d941dfb0567 100644
--- a/drivers/firmware/imx/se_ctrl.h
+++ b/drivers/firmware/imx/se_ctrl.h
@@ -8,17 +8,28 @@
#include <linux/bitfield.h>
#include <linux/miscdevice.h>
-#include <linux/semaphore.h>
#include <linux/mailbox_client.h>
+#include <linux/semaphore.h>
+#include <linux/workqueue.h>
#define MAX_FW_LOAD_RETRIES 50
#define SE_MSG_WORD_SZ 0x4
#define RES_STATUS(x) FIELD_GET(0x000000ff, x)
+#define MAX_DATA_SIZE_PER_USER (128 * 1024)
#define MAX_NVM_MSG_LEN (256)
#define MESSAGING_VERSION_6 0x6
#define MESSAGING_VERSION_7 0x7
+struct se_if_open_gate {
+ struct miscdevice miscdev;
+ struct se_if_priv *priv;
+ /* to lock to update the structure */
+ struct mutex lock;
+ struct kref refcount;
+ bool dying;
+};
+
struct se_clbk_handle {
struct se_if_device_ctx *dev_ctx;
struct completion done;
@@ -45,10 +56,43 @@ struct se_imem_buf {
u32 state;
};
+struct se_buf_desc {
+ u8 *shared_buf_ptr;
+ void __user *usr_buf_ptr;
+ u32 size;
+ struct list_head link;
+};
+
+struct se_shared_mem {
+ dma_addr_t dma_addr;
+ u32 size;
+ u32 pos;
+ u8 *ptr;
+};
+
+struct se_shared_mem_mgmt_info {
+ struct list_head pending_in;
+ struct list_head pending_out;
+
+ struct se_shared_mem non_secure_mem;
+};
+
/* Private struct for each char device instance. */
struct se_if_device_ctx {
struct se_if_priv *priv;
+ struct miscdevice *miscdev;
const char *devname;
+ bool cleanup_done;
+ unsigned long rcv_msg_timeout_jiffies;
+
+ /* process one file operation at a time. */
+ struct mutex fops_lock;
+
+ struct se_shared_mem_mgmt_info se_shared_mem_mgmt;
+ struct list_head link;
+
+ /* Add reference counting */
+ struct kref refcount;
};
/* Header of the messages exchange with the EdgeLock Enclave */
@@ -59,7 +103,8 @@ struct se_msg_hdr {
u8 tag;
} __packed;
-#define SE_MU_HDR_SZ 4
+#define SE_MU_HDR_SZ 4
+#define SE_MU_HDR_WORD_SZ 1
struct se_api_msg {
struct se_msg_hdr header;
@@ -112,9 +157,34 @@ struct se_if_priv {
struct se_fw_load_info load_fw;
atomic_t fw_busy;
+ /*
+ * Serialise the fw_busy_dev_ctx and fw_busy state updates between the
+ * timeout path, late-response callback/work, and teardown.
+ */
+ spinlock_t fw_busy_lock;
+ struct se_if_device_ctx *fw_busy_dev_ctx;
+ struct work_struct fw_busy_work;
struct se_if_device_ctx *priv_dev_ctx;
+ struct list_head dev_ctx_list;
+
+ /* prevent modifying priv member variable in parallel. */
+ struct mutex modify_lock;
+ u32 active_devctx_count;
+ u32 dev_ctx_mono_count;
+
+ /* Add reference counting */
+ struct kref refcount;
+
+ /* stable gate used by .open() */
+ struct se_if_open_gate *open_gate;
};
char *get_se_if_name(u8 se_if_id);
+void unset_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx);
+int set_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx);
+bool se_is_fw_busy_ctx(struct se_if_device_ctx *dev_ctx);
+void se_dev_ctx_shared_mem_cleanup(struct se_if_device_ctx *dev_ctx);
+int get_shared_mem_slot(struct se_if_device_ctx *dev_ctx,
+ u32 *length, dma_addr_t *ele_dma_addr, void **ptr);
#endif
diff --git a/include/uapi/linux/se_ioctl.h b/include/uapi/linux/se_ioctl.h
new file mode 100644
index 000000000000..ea14cec05020
--- /dev/null
+++ b/include/uapi/linux/se_ioctl.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: (GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause*/
+/*
+ * Copyright 2025 NXP
+ */
+
+#ifndef SE_IOCTL_H
+#define SE_IOCTL_H
+
+#include <linux/types.h>
+
+#define SE_TYPE_STR_DBG "dbg"
+#define SE_TYPE_STR_HSM "hsm"
+#define SE_TYPE_ID_UNKWN 0x0
+#define SE_TYPE_ID_DBG 0x1
+#define SE_TYPE_ID_HSM 0x2
+/* IOCTL definitions. */
+
+struct se_ioctl_setup_iobuf {
+ __u64 user_buf;
+ __u32 length;
+ __u32 flags;
+ __u64 ele_addr;
+};
+
+struct se_ioctl_shared_mem_cfg {
+ __u32 base_offset;
+ __u32 size;
+};
+
+struct se_ioctl_get_if_info {
+ __u8 se_if_id;
+ __u8 interrupt_idx;
+ __u8 tz;
+ __u8 did;
+ __u8 cmd_tag;
+ __u8 rsp_tag;
+ __u8 success_tag;
+ __u8 base_api_ver;
+ __u8 fw_api_ver;
+};
+
+struct se_ioctl_cmd_snd_rcv_rsp_info {
+ __u64 tx_buf;
+ __u32 tx_buf_sz;
+ __u64 rx_buf;
+ __u32 rx_buf_sz;
+};
+
+struct se_ioctl_get_soc_info {
+ __u16 soc_id;
+ __u16 soc_rev;
+};
+
+/* IO Buffer Flags */
+#define SE_IO_BUF_FLAGS_IS_OUTPUT (0x00u)
+#define SE_IO_BUF_FLAGS_IS_INPUT (0x01u)
+#define SE_IO_BUF_FLAGS_USE_SEC_MEM (0x02u)
+#define SE_IO_BUF_FLAGS_USE_SHORT_ADDR (0x04u)
+#define SE_IO_BUF_FLAGS_IS_IN_OUT (0x10u)
+
+/* IOCTLS */
+#define SE_IOCTL 0x0A /* like MISC_MAJOR. */
+
+/*
+ * ioctl to designated the current fd as logical-reciever.
+ * This is ioctl is send when the nvm-daemon, a slave to the
+ * firmware is started by the user.
+ */
+#define SE_IOCTL_ENABLE_CMD_RCV _IO(SE_IOCTL, 0x01)
+
+/*
+ * ioctl to get the buffer allocated from the memory, which is shared
+ * between kernel and FW.
+ * Post allocation, the kernel tagged the allocated memory with:
+ * Output
+ * Input
+ * Input-Output
+ * Short address
+ * Secure-memory
+ */
+#define SE_IOCTL_SETUP_IOBUF _IOWR(SE_IOCTL, 0x03, struct se_ioctl_setup_iobuf)
+
+/*
+ * ioctl to get the mu information, that is used to exchange message
+ * with FW, from user-spaced.
+ */
+#define SE_IOCTL_GET_MU_INFO _IOR(SE_IOCTL, 0x04, struct se_ioctl_get_if_info)
+/*
+ * ioctl to get SoC Info from user-space.
+ */
+#define SE_IOCTL_GET_SOC_INFO _IOR(SE_IOCTL, 0x06, struct se_ioctl_get_soc_info)
+
+/*
+ * ioctl to send command and receive response from user-space.
+ */
+#define SE_IOCTL_CMD_SEND_RCV_RSP _IOWR(SE_IOCTL, 0x07, struct se_ioctl_cmd_snd_rcv_rsp_info)
+#endif
--
2.43.0
^ permalink raw reply related
* [PATCH v28 4/7] firmware: imx: device context dedicated to priv
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel,
sashiko-bot
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
Add priv_dev_ctx to prepare enabling misc-device context based send-receive
path, to communicate with FW.
No functionality change.
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
Changes from v27 to v28
- Fix init_misc_device_context() to return 0 on the success path.
The helper initialized ret to -ENOMEM and returned ret even after
successfully allocating and initializing priv_dev_ctx. This caused
se_if_probe() to treat a successful init_misc_device_context() call as a
failure and abort probe. Return 0 after assigning *new_dev_ctx.
- Avoid a possible NULL pointer dereference in se_if_rx_callback() after
waking the synchronous response waiter.
The response callback used se_clbk_hdl->dev_ctx->devname after calling
complete() and dropping clbk_rx_lock. The awakened ele_msg_send_rcv()
cleanup path can clear waiting_rsp_clbk_hdl.dev_ctx under the same lock
before the size-mismatch dev_err() is emitted.
Snapshot devname while clbk_rx_lock is still held and use the local copy
after dropping the lock.
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260715-imx-se-if-v27-0-bb7c45952f06@nxp.com?part=4
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
---
drivers/firmware/imx/ele_base_msg.c | 15 ++++++-----
drivers/firmware/imx/ele_common.c | 53 ++++++++++++++++++++++---------------
drivers/firmware/imx/ele_common.h | 8 +++---
drivers/firmware/imx/se_ctrl.c | 42 +++++++++++++++++++++++++++++
drivers/firmware/imx/se_ctrl.h | 9 +++++++
5 files changed, 94 insertions(+), 33 deletions(-)
diff --git a/drivers/firmware/imx/ele_base_msg.c b/drivers/firmware/imx/ele_base_msg.c
index ac6a1c1e8cd4..6a6db60887af 100644
--- a/drivers/firmware/imx/ele_base_msg.c
+++ b/drivers/firmware/imx/ele_base_msg.c
@@ -66,8 +66,9 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info)
tx_msg->data[0] = upper_32_bits(get_info_addr);
tx_msg->data[1] = lower_32_bits(get_info_addr);
tx_msg->data[2] = sizeof(*s_info);
- ret = ele_msg_send_rcv(priv, tx_msg, ELE_GET_INFO_REQ_MSG_SZ, rx_msg,
- ELE_GET_INFO_RSP_MSG_SZ);
+
+ ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_GET_INFO_REQ_MSG_SZ,
+ rx_msg, ELE_GET_INFO_RSP_MSG_SZ);
if (ret < 0) {
ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
return ret;
@@ -112,8 +113,8 @@ int ele_ping(struct se_if_priv *priv)
se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header,
ELE_PING_REQ, ELE_PING_REQ_SZ, true);
- ret = ele_msg_send_rcv(priv, tx_msg, ELE_PING_REQ_SZ, rx_msg,
- ELE_PING_RSP_SZ);
+ ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_PING_REQ_SZ,
+ rx_msg, ELE_PING_RSP_SZ);
if (ret < 0)
return ret;
@@ -160,7 +161,7 @@ int ele_service_swap(struct se_if_priv *priv,
if (ret)
return -EINVAL;
- ret = ele_msg_send_rcv(priv, tx_msg, ELE_SERVICE_SWAP_REQ_MSG_SZ,
+ ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_SERVICE_SWAP_REQ_MSG_SZ,
rx_msg, ELE_SERVICE_SWAP_RSP_MSG_SZ);
if (ret < 0)
return ret;
@@ -208,7 +209,7 @@ int ele_fw_authenticate(struct se_if_priv *priv, dma_addr_t contnr_addr,
tx_msg->data[1] = 0;
tx_msg->data[2] = lower_32_bits(img_addr);
- ret = ele_msg_send_rcv(priv, tx_msg, ELE_FW_AUTH_REQ_SZ, rx_msg,
+ ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_FW_AUTH_REQ_SZ, rx_msg,
ELE_FW_AUTH_RSP_MSG_SZ);
if (ret < 0)
return ret;
@@ -246,7 +247,7 @@ int ele_debug_dump(struct se_if_priv *priv)
do {
memset(rx_msg, 0x0, ELE_DEBUG_DUMP_RSP_SZ);
- ret = ele_msg_send_rcv(priv, tx_msg, ELE_DEBUG_DUMP_REQ_SZ,
+ ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_DEBUG_DUMP_REQ_SZ,
rx_msg, ELE_DEBUG_DUMP_RSP_SZ);
if (ret < 0)
return ret;
diff --git a/drivers/firmware/imx/ele_common.c b/drivers/firmware/imx/ele_common.c
index ca8a32a82689..1ca22c53b55c 100644
--- a/drivers/firmware/imx/ele_common.c
+++ b/drivers/firmware/imx/ele_common.c
@@ -46,8 +46,9 @@ int se_update_msg_chksum(u32 *msg, u32 msg_len)
return 0;
}
-int ele_msg_rcv(struct se_if_priv *priv, struct se_clbk_handle *se_clbk_hdl)
+int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk_hdl)
{
+ struct se_if_priv *priv = dev_ctx->priv;
bool wait_uninterruptible = false;
unsigned long remaining_jiffies;
unsigned long flags;
@@ -111,7 +112,7 @@ int ele_msg_rcv(struct se_if_priv *priv, struct se_clbk_handle *se_clbk_hdl)
return ret;
}
-int ele_msg_send(struct se_if_priv *priv,
+int ele_msg_send(struct se_if_device_ctx *dev_ctx,
void *tx_msg,
int tx_msg_sz)
{
@@ -123,9 +124,9 @@ int ele_msg_send(struct se_if_priv *priv,
* carried in the message.
*/
if (header->size << 2 != tx_msg_sz) {
- dev_err(priv->dev,
- "User buf hdr: 0x%x, sz mismatced with input-sz (%d != %d).",
- *(u32 *)header, header->size << 2, tx_msg_sz);
+ dev_err(dev_ctx->priv->dev,
+ "%s: User buf hdr: 0x%x, sz mismatched with input-sz (%d != %d).",
+ dev_ctx->devname, *(u32 *)header, header->size << 2, tx_msg_sz);
return -EINVAL;
}
@@ -135,9 +136,10 @@ int ele_msg_send(struct se_if_priv *priv,
* caller-provided tx_msg pointer after mbox_send_message() returns, so
* the caller-owned buffer may be released after a successful send.
*/
- err = mbox_send_message(priv->tx_chan, tx_msg);
+ err = mbox_send_message(dev_ctx->priv->tx_chan, tx_msg);
if (err < 0) {
- dev_err(priv->dev, "Error: mbox_send_message failure.\n");
+ dev_err(dev_ctx->priv->dev,
+ "%s: Error: mbox_send_message failure.", dev_ctx->devname);
return err;
}
@@ -149,34 +151,38 @@ static void ele_msg_send_rcv_cleanup(struct se_if_priv *priv)
unsigned long flags;
spin_lock_irqsave(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags);
+ priv->waiting_rsp_clbk_hdl.dev_ctx = NULL;
priv->waiting_rsp_clbk_hdl.rx_msg = NULL;
priv->waiting_rsp_clbk_hdl.rx_msg_sz = 0;
spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags);
}
/* API used for send/receive blocking call. */
-int ele_msg_send_rcv(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz,
- void *rx_msg, int exp_rx_msg_sz)
+int ele_msg_send_rcv(struct se_if_device_ctx *dev_ctx, void *tx_msg,
+ int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz)
{
+ struct se_if_priv *priv = dev_ctx->priv;
+ unsigned long flags;
int err;
guard(mutex)(&priv->se_if_cmd_lock);
if (atomic_read(&priv->fw_busy)) {
- dev_dbg(priv->dev, "ELE became unresponsive.\n");
+ dev_dbg(priv->dev, "%s: ELE became unresponsive.\n", dev_ctx->devname);
return -EBUSY;
}
reinit_completion(&priv->waiting_rsp_clbk_hdl.done);
+ priv->waiting_rsp_clbk_hdl.dev_ctx = dev_ctx;
priv->waiting_rsp_clbk_hdl.rx_msg_sz = exp_rx_msg_sz;
priv->waiting_rsp_clbk_hdl.rx_msg = rx_msg;
- err = ele_msg_send(priv, tx_msg, tx_msg_sz);
+ err = ele_msg_send(dev_ctx, tx_msg, tx_msg_sz);
if (err < 0) {
ele_msg_send_rcv_cleanup(priv);
return err;
}
- err = ele_msg_rcv(priv, &priv->waiting_rsp_clbk_hdl);
+ err = ele_msg_rcv(dev_ctx, &priv->waiting_rsp_clbk_hdl);
if (priv->waiting_rsp_clbk_hdl.signal_rcvd) {
/*
@@ -189,7 +195,8 @@ int ele_msg_send_rcv(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz,
if (err > 0)
err = -ERESTARTSYS;
priv->waiting_rsp_clbk_hdl.signal_rcvd = false;
- dev_err(priv->dev, "Err[0x%x]:Interrupted by signal.", err);
+ dev_err(priv->dev, "%s: Err[0x%x]:Interrupted by signal.",
+ dev_ctx->devname, err);
}
ele_msg_send_rcv_cleanup(priv);
@@ -240,7 +247,7 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
if (header->tag == priv->if_defs->cmd_tag) {
se_clbk_hdl = &priv->cmd_receiver_clbk_hdl;
spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
- if (!se_clbk_hdl->rx_msg) {
+ if (!se_clbk_hdl->dev_ctx || !se_clbk_hdl->rx_msg) {
spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
dev_warn(dev, "No command receiver registered for message: %.8x\n",
*((u32 *)header));
@@ -254,8 +261,8 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
* SE_IOCTL_ENABLE_CMD_RCV and is not subject to the timeout/circuit-
* breaker handling used for rsp_tag messages.
*/
- dev_dbg(dev, "Selecting cmd receiver: for mesg header:0x%x.",
- *(u32 *)header);
+ dev_dbg(dev, "Selecting cmd receiver:%s for mesg header:0x%x.",
+ se_clbk_hdl->dev_ctx->devname, *(u32 *)header);
/*
* Pre-allocated buffer of MAX_NVM_MSG_LEN
@@ -275,12 +282,13 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
if (sz_mismatch)
dev_err(dev,
- "CMD-RCVER NVM: hdr(0x%x) with different sz(%d != %d).\n",
- *(u32 *)header,
+ "%s: CMD-RCVER NVM: hdr(0x%x) with different sz(%d != %d).\n",
+ se_clbk_hdl->dev_ctx->devname, *(u32 *)header,
(header->size << 2), rx_msg_sz);
} else if (header->tag == priv->if_defs->rsp_tag) {
bool exception_for_sz_mismatch = check_hdr_exception_for_sz(priv, header);
u32 exp_rx_msg_sz = 0;
+ const char *devname = NULL;
/*
* waiting_rsp_clbk_hdl.rx_msg is owned by the synchronous sender in
@@ -302,8 +310,8 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
dev_info(dev, "ELE responded (late), recovery FW available.");
return;
}
- dev_dbg(dev, "Selecting resp waiter: for mesg header:0x%x.",
- *(u32 *)header);
+ dev_dbg(dev, "Selecting resp waiter:%s for mesg header:0x%x.",
+ se_clbk_hdl->dev_ctx->devname, *(u32 *)header);
/*
* For rsp_tag traffic, the sender provides the expected response
@@ -317,14 +325,15 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
se_clbk_hdl->rx_msg_sz = min(rx_msg_sz, exp_rx_msg_sz);
}
+ devname = se_clbk_hdl->dev_ctx->devname;
memcpy(se_clbk_hdl->rx_msg, msg, se_clbk_hdl->rx_msg_sz);
complete(&se_clbk_hdl->done);
spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
if (sz_mismatch)
dev_err(dev,
- "Rsp to CMD: hdr(0x%x) with different sz(%d != %d).\n",
- *(u32 *)header,
+ "%s: Rsp to CMD: hdr(0x%x) with different sz(%d != %d).\n",
+ devname, *(u32 *)header,
(header->size << 2), exp_rx_msg_sz);
} else {
dev_err(dev, "Failed to select a device for message: %.8x\n",
diff --git a/drivers/firmware/imx/ele_common.h b/drivers/firmware/imx/ele_common.h
index 42460d1d5dec..f7d7e3e62dd4 100644
--- a/drivers/firmware/imx/ele_common.h
+++ b/drivers/firmware/imx/ele_common.h
@@ -14,12 +14,12 @@
int se_update_msg_chksum(u32 *msg, u32 msg_len);
-int ele_msg_rcv(struct se_if_priv *priv, struct se_clbk_handle *se_clbk_hdl);
+int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk_hdl);
-int ele_msg_send(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz);
+int ele_msg_send(struct se_if_device_ctx *dev_ctx, void *tx_msg, int tx_msg_sz);
-int ele_msg_send_rcv(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz,
- void *rx_msg, int exp_rx_msg_sz);
+int ele_msg_send_rcv(struct se_if_device_ctx *dev_ctx, void *tx_msg,
+ int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz);
void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg);
diff --git a/drivers/firmware/imx/se_ctrl.c b/drivers/firmware/imx/se_ctrl.c
index 9bb59c200911..f43d06ab123b 100644
--- a/drivers/firmware/imx/se_ctrl.c
+++ b/drivers/firmware/imx/se_ctrl.c
@@ -209,6 +209,36 @@ static int get_se_soc_info(struct se_if_priv *priv, const struct se_soc_info *se
return 0;
}
+static int init_misc_device_context(struct se_if_priv *priv, int ch_id,
+ struct se_if_device_ctx **new_dev_ctx)
+{
+ const char *err_str = "Failed to allocate memory";
+ struct se_if_device_ctx *dev_ctx;
+ int ret = -ENOMEM;
+
+ dev_ctx = kzalloc_obj(*dev_ctx, GFP_KERNEL);
+
+ if (!dev_ctx)
+ return ret;
+
+ dev_ctx->devname = kasprintf(GFP_KERNEL, "%s0_ch%d",
+ get_se_if_name(priv->if_defs->se_if_type),
+ ch_id);
+ if (!dev_ctx->devname)
+ goto exit;
+
+ dev_ctx->priv = priv;
+ *new_dev_ctx = dev_ctx;
+
+ return 0;
+exit:
+ *new_dev_ctx = NULL;
+
+ kfree(dev_ctx->devname);
+ kfree(dev_ctx);
+ return dev_err_probe(priv->dev, ret, "%s", err_str);
+}
+
/* interface for managed res to free a mailbox channel */
static void if_mbox_free_channel(void *mbox_chan)
{
@@ -265,6 +295,12 @@ static void se_if_probe_cleanup(void *plat_dev)
of_reserved_mem_device_release(dev);
dev_set_drvdata(dev, NULL);
+
+ if (priv->priv_dev_ctx) {
+ kfree(priv->priv_dev_ctx->devname);
+ kfree(priv->priv_dev_ctx);
+ priv->priv_dev_ctx = NULL;
+ }
kfree(priv);
}
@@ -335,6 +371,12 @@ static int se_if_probe(struct platform_device *pdev)
if (ret)
return dev_err_probe(dev, ret, "Failed to set 32-bit coherent DMA mask.");
+ ret = init_misc_device_context(priv, 0, &priv->priv_dev_ctx);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed[0x%x] to create device contexts.",
+ ret);
+
if (if_node->if_defs.se_if_type == SE_TYPE_ID_HSM) {
ret = get_se_soc_info(priv, se_info);
if (ret)
diff --git a/drivers/firmware/imx/se_ctrl.h b/drivers/firmware/imx/se_ctrl.h
index b6833abd81d0..355d38684b1b 100644
--- a/drivers/firmware/imx/se_ctrl.h
+++ b/drivers/firmware/imx/se_ctrl.h
@@ -20,6 +20,7 @@
#define MESSAGING_VERSION_7 0x7
struct se_clbk_handle {
+ struct se_if_device_ctx *dev_ctx;
struct completion done;
bool signal_rcvd;
u32 rx_msg_sz;
@@ -44,6 +45,12 @@ struct se_imem_buf {
u32 state;
};
+/* Private struct for each char device instance. */
+struct se_if_device_ctx {
+ struct se_if_priv *priv;
+ const char *devname;
+};
+
/* Header of the messages exchange with the EdgeLock Enclave */
struct se_msg_hdr {
u8 ver;
@@ -105,6 +112,8 @@ struct se_if_priv {
struct se_fw_load_info load_fw;
atomic_t fw_busy;
+
+ struct se_if_device_ctx *priv_dev_ctx;
};
char *get_se_if_name(u8 se_if_id);
--
2.43.0
^ permalink raw reply related
* [PATCH v28 3/7] firmware: imx: add driver for NXP EdgeLock Enclave
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel,
Frieder Schrempf, sashiko-bot
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
Add MU-based communication interface for secure enclave.
NXP hardware IP(s) for secure-enclaves like Edgelock Enclave(ELE), are
embedded in the SoC to support the features like HSM, SHE & V2X, using
message based communication interface.
The secure enclave FW communicates with Linux over single or multiple
dedicated messaging unit(MU) based interface(s).
Exists on i.MX SoC(s) like i.MX8ULP, i.MX93, i.MX95 etc.
For i.MX9x SoC(s) there is at least one dedicated ELE MU(s) for each
world - Linux(one or more) and OPTEE-OS (one or more).
Other dependent kernel drivers will be:
- NVMEM: that supports non-volatile devices like EFUSES,
managed by NXP's secure-enclave.
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: Frieder Schrempf <frieder.schrempf@kontron.de>
Reviewed-by: Frieder Schrempf <frieder.schrempf@kontron.de>
---
Changes from v27 to v28:
Fix several issues reported by Sashiko in the ELE driver:
- Add cleanup helper for ele_get_info() and remove the goto-based cleanup path
that mixed manual cleanup with scoped __free(kfree) objects.
- Document the mailbox TX buffer lifetime assumption in ele_msg_send().
The i.MX MU mailbox controller copies message payload words into MU
registers synchronously and does not retain the caller-provided tx_msg
pointer after mbox_send_message() returns.
- Replace se_get_msg_chksum() with se_update_msg_chksum(), which validates
the message pointer and size, calculates the checksum, and updates the
checksum word directly.
- Remove the stale ret check after se_fill_cmd_msg_hdr(), since
se_fill_cmd_msg_hdr() now returns void.
- Validate the encrypted IMEM export size returned by firmware against
ELE_IMEM_SIZE before storing it in imem->size for later resume-time import.
- Add a mutex to serialize population of common SoC-level information stored
in var_se_info.
- Add NULL checks for devm_kasprintf() results before passing revision and
serial-number strings to soc_device_register().
- Set a 32-bit DMA/coherent mask to make the ELE 32-bit firmware address
constraint explicit, while still relying on the reserved memory region for
ELE-accessible DMA memory.
- Keep mutable firmware-load and IMEM state per device in struct se_if_priv,
while var_se_info only caches immutable SoC-level revision data.
- Keep the SoC device unregister devres action and devm-managed IMEM coherent
memory handling from v27.
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260715-imx-se-if-v27-0-bb7c45952f06@nxp.com?part=3
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
---
drivers/firmware/imx/Kconfig | 13 +
drivers/firmware/imx/Makefile | 2 +
drivers/firmware/imx/ele_base_msg.c | 277 +++++++++++++++++++++
drivers/firmware/imx/ele_base_msg.h | 98 ++++++++
drivers/firmware/imx/ele_common.c | 470 ++++++++++++++++++++++++++++++++++++
drivers/firmware/imx/ele_common.h | 43 ++++
drivers/firmware/imx/se_ctrl.c | 417 ++++++++++++++++++++++++++++++++
drivers/firmware/imx/se_ctrl.h | 111 +++++++++
include/linux/firmware/imx/se_api.h | 14 ++
9 files changed, 1445 insertions(+)
diff --git a/drivers/firmware/imx/Kconfig b/drivers/firmware/imx/Kconfig
index 127ad752acf8..e3cb7f965e70 100644
--- a/drivers/firmware/imx/Kconfig
+++ b/drivers/firmware/imx/Kconfig
@@ -55,3 +55,16 @@ config IMX_SCMI_MISC_DRV
core that could provide misc functions such as board control.
This driver can also be built as a module.
+
+config IMX_SEC_ENCLAVE
+ tristate "i.MX Embedded Secure Enclave - EdgeLock Enclave Firmware driver."
+ depends on MAILBOX && ((IMX_MBOX && ARCH_MXC && ARM64) || COMPILE_TEST)
+ select FW_LOADER
+ default m if ARCH_MXC
+
+ help
+ Exposes APIs supported by the iMX Secure Enclave HW IP called:
+ - EdgeLock Enclave Firmware (for i.MX8ULP, i.MX93),
+ like base, HSM, V2X & SHE using the SAB protocol via the shared Messaging
+ Unit. This driver exposes these interfaces via a set of file descriptors
+ allowing to configure shared memory, send and receive messages.
diff --git a/drivers/firmware/imx/Makefile b/drivers/firmware/imx/Makefile
index 3bbaffa6e347..4412b15846b1 100644
--- a/drivers/firmware/imx/Makefile
+++ b/drivers/firmware/imx/Makefile
@@ -4,3 +4,5 @@ obj-$(CONFIG_IMX_SCU) += imx-scu.o misc.o imx-scu-irq.o rm.o imx-scu-soc.o
obj-${CONFIG_IMX_SCMI_CPU_DRV} += sm-cpu.o
obj-${CONFIG_IMX_SCMI_MISC_DRV} += sm-misc.o
obj-${CONFIG_IMX_SCMI_LMM_DRV} += sm-lmm.o
+sec_enclave-objs = se_ctrl.o ele_common.o ele_base_msg.o
+obj-${CONFIG_IMX_SEC_ENCLAVE} += sec_enclave.o
diff --git a/drivers/firmware/imx/ele_base_msg.c b/drivers/firmware/imx/ele_base_msg.c
new file mode 100644
index 000000000000..ac6a1c1e8cd4
--- /dev/null
+++ b/drivers/firmware/imx/ele_base_msg.c
@@ -0,0 +1,277 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2025 NXP
+ */
+
+#include <linux/types.h>
+
+#include <linux/cleanup.h>
+#include <linux/completion.h>
+#include <linux/dma-mapping.h>
+#include <linux/genalloc.h>
+
+#include "ele_base_msg.h"
+#include "ele_common.h"
+
+#define FW_DBG_DUMP_FIXED_STR "ELE"
+
+static void ele_get_info_cleanup(struct se_if_priv *priv, u32 *buf, dma_addr_t d_addr,
+ size_t size)
+{
+ if (priv->mem_pool)
+ gen_pool_free(priv->mem_pool, (unsigned long)buf, size);
+ else
+ dma_free_coherent(priv->dev, size, buf, d_addr);
+}
+
+int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info)
+{
+ dma_addr_t get_info_addr = 0;
+ u32 *get_info_data = NULL;
+ int ret = 0;
+
+ if (!priv)
+ return -EINVAL;
+
+ memset(s_info, 0x0, sizeof(*s_info));
+
+ struct se_api_msg *tx_msg __free(kfree) =
+ kzalloc(ELE_GET_INFO_REQ_MSG_SZ, GFP_KERNEL);
+ if (!tx_msg)
+ return -ENOMEM;
+
+ struct se_api_msg *rx_msg __free(kfree) =
+ kzalloc(ELE_GET_INFO_RSP_MSG_SZ, GFP_KERNEL);
+ if (!rx_msg)
+ return -ENOMEM;
+
+ if (priv->mem_pool)
+ get_info_data = gen_pool_dma_alloc(priv->mem_pool,
+ ELE_GET_INFO_BUFF_SZ,
+ &get_info_addr);
+ else
+ get_info_data = dma_alloc_coherent(priv->dev,
+ ELE_GET_INFO_BUFF_SZ,
+ &get_info_addr,
+ GFP_KERNEL);
+ if (!get_info_data) {
+ dev_err(priv->dev,
+ "%s: Failed to allocate get_info_addr.", __func__);
+ return -ENOMEM;
+ }
+
+ se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header,
+ ELE_GET_INFO_REQ, ELE_GET_INFO_REQ_MSG_SZ, true);
+
+ tx_msg->data[0] = upper_32_bits(get_info_addr);
+ tx_msg->data[1] = lower_32_bits(get_info_addr);
+ tx_msg->data[2] = sizeof(*s_info);
+ ret = ele_msg_send_rcv(priv, tx_msg, ELE_GET_INFO_REQ_MSG_SZ, rx_msg,
+ ELE_GET_INFO_RSP_MSG_SZ);
+ if (ret < 0) {
+ ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
+ return ret;
+ }
+
+ ret = se_val_rsp_hdr_n_status(priv, rx_msg, ELE_GET_INFO_REQ,
+ ELE_GET_INFO_RSP_MSG_SZ, true);
+ if (ret < 0) {
+ ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
+ return ret;
+ }
+
+ memcpy(s_info, get_info_data, sizeof(*s_info));
+
+ ele_get_info_cleanup(priv, get_info_data, get_info_addr, ELE_GET_INFO_BUFF_SZ);
+
+ return ret;
+}
+
+int ele_fetch_soc_info(struct se_if_priv *priv, void *data)
+{
+ return ele_get_info(priv, data);
+}
+
+int ele_ping(struct se_if_priv *priv)
+{
+ int ret = 0;
+
+ if (!priv)
+ return -EINVAL;
+
+ struct se_api_msg *tx_msg __free(kfree) = kzalloc(ELE_PING_REQ_SZ,
+ GFP_KERNEL);
+ if (!tx_msg)
+ return -ENOMEM;
+
+ struct se_api_msg *rx_msg __free(kfree) = kzalloc(ELE_PING_RSP_SZ,
+ GFP_KERNEL);
+ if (!rx_msg)
+ return -ENOMEM;
+
+ se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header,
+ ELE_PING_REQ, ELE_PING_REQ_SZ, true);
+
+ ret = ele_msg_send_rcv(priv, tx_msg, ELE_PING_REQ_SZ, rx_msg,
+ ELE_PING_RSP_SZ);
+ if (ret < 0)
+ return ret;
+
+ ret = se_val_rsp_hdr_n_status(priv, rx_msg, ELE_PING_REQ,
+ ELE_PING_RSP_SZ, true);
+
+ return ret;
+}
+
+int ele_service_swap(struct se_if_priv *priv,
+ dma_addr_t addr,
+ u32 addr_size, u16 flag)
+{
+ int ret = 0;
+
+ if (upper_32_bits(addr)) {
+ dev_err(priv->dev,
+ "ELE service-swap address exceeds 32-bit range: %pad\n",
+ &addr);
+ return -ERANGE;
+ }
+
+ if (!priv)
+ return -EINVAL;
+
+ struct se_api_msg *tx_msg __free(kfree) =
+ kzalloc(ELE_SERVICE_SWAP_REQ_MSG_SZ, GFP_KERNEL);
+ if (!tx_msg)
+ return -ENOMEM;
+
+ struct se_api_msg *rx_msg __free(kfree) =
+ kzalloc(ELE_SERVICE_SWAP_RSP_MSG_SZ, GFP_KERNEL);
+ if (!rx_msg)
+ return -ENOMEM;
+
+ se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header,
+ ELE_SERVICE_SWAP_REQ, ELE_SERVICE_SWAP_REQ_MSG_SZ, true);
+
+ tx_msg->data[0] = flag;
+ tx_msg->data[1] = addr_size;
+ tx_msg->data[2] = ELE_NONE_VAL;
+ tx_msg->data[3] = lower_32_bits(addr);
+ ret = se_update_msg_chksum((u32 *)&tx_msg[0], ELE_SERVICE_SWAP_REQ_MSG_SZ);
+ if (ret)
+ return -EINVAL;
+
+ ret = ele_msg_send_rcv(priv, tx_msg, ELE_SERVICE_SWAP_REQ_MSG_SZ,
+ rx_msg, ELE_SERVICE_SWAP_RSP_MSG_SZ);
+ if (ret < 0)
+ return ret;
+
+ ret = se_val_rsp_hdr_n_status(priv, rx_msg, ELE_SERVICE_SWAP_REQ,
+ ELE_SERVICE_SWAP_RSP_MSG_SZ, true);
+ if (ret)
+ return ret;
+
+ if (flag == ELE_IMEM_EXPORT)
+ ret = rx_msg->data[1];
+ else
+ ret = 0;
+
+ return ret;
+}
+
+int ele_fw_authenticate(struct se_if_priv *priv, dma_addr_t contnr_addr,
+ dma_addr_t img_addr)
+{
+ int ret = 0;
+
+ if (!priv)
+ return -EINVAL;
+
+ if (upper_32_bits(contnr_addr) || upper_32_bits(img_addr)) {
+ dev_err(priv->dev, "Wrong address: %pap %pap\n", &contnr_addr, &img_addr);
+ return -EINVAL;
+ }
+
+ struct se_api_msg *tx_msg __free(kfree) =
+ kzalloc(ELE_FW_AUTH_REQ_SZ, GFP_KERNEL);
+ if (!tx_msg)
+ return -ENOMEM;
+
+ struct se_api_msg *rx_msg __free(kfree) =
+ kzalloc(ELE_FW_AUTH_RSP_MSG_SZ, GFP_KERNEL);
+ if (!rx_msg)
+ return -ENOMEM;
+
+ se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header,
+ ELE_FW_AUTH_REQ, ELE_FW_AUTH_REQ_SZ, true);
+
+ tx_msg->data[0] = lower_32_bits(contnr_addr);
+ tx_msg->data[1] = 0;
+ tx_msg->data[2] = lower_32_bits(img_addr);
+
+ ret = ele_msg_send_rcv(priv, tx_msg, ELE_FW_AUTH_REQ_SZ, rx_msg,
+ ELE_FW_AUTH_RSP_MSG_SZ);
+ if (ret < 0)
+ return ret;
+
+ ret = se_val_rsp_hdr_n_status(priv, rx_msg, ELE_FW_AUTH_REQ,
+ ELE_FW_AUTH_RSP_MSG_SZ, true);
+
+ return ret;
+}
+
+int ele_debug_dump(struct se_if_priv *priv)
+{
+ bool keep_logging;
+ int msg_ex_cnt;
+ int ret = 0;
+ int i;
+
+ if (!priv)
+ return -EINVAL;
+
+ struct se_api_msg *tx_msg __free(kfree) = kzalloc(ELE_DEBUG_DUMP_REQ_SZ,
+ GFP_KERNEL);
+ if (!tx_msg)
+ return -ENOMEM;
+
+ struct se_api_msg *rx_msg __free(kfree) = kzalloc(ELE_DEBUG_DUMP_RSP_SZ,
+ GFP_KERNEL);
+ if (!rx_msg)
+ return -ENOMEM;
+
+ se_fill_cmd_msg_hdr(priv, &tx_msg->header, ELE_DEBUG_DUMP_REQ,
+ ELE_DEBUG_DUMP_REQ_SZ, true);
+
+ msg_ex_cnt = 0;
+ do {
+ memset(rx_msg, 0x0, ELE_DEBUG_DUMP_RSP_SZ);
+
+ ret = ele_msg_send_rcv(priv, tx_msg, ELE_DEBUG_DUMP_REQ_SZ,
+ rx_msg, ELE_DEBUG_DUMP_RSP_SZ);
+ if (ret < 0)
+ return ret;
+
+ ret = se_val_rsp_hdr_n_status(priv, rx_msg, ELE_DEBUG_DUMP_REQ,
+ ELE_DEBUG_DUMP_RSP_SZ, true);
+ if (ret) {
+ dev_err(priv->dev, "Dump_Debug_Buffer Error: %x.", ret);
+ break;
+ }
+ keep_logging = (rx_msg->header.size >= (ELE_DEBUG_DUMP_RSP_SZ >> 2) &&
+ msg_ex_cnt < ELE_MAX_DBG_DMP_PKT);
+
+ rx_msg->header.size -= 2;
+
+ if (rx_msg->header.size > 2)
+ rx_msg->header.size--;
+
+ for (i = 0; i < rx_msg->header.size; i += 2)
+ dev_info(priv->dev, "%s%02x_%02x: 0x%08x 0x%08x",
+ FW_DBG_DUMP_FIXED_STR, msg_ex_cnt, i,
+ rx_msg->data[i + 1], rx_msg->data[i + 2]);
+
+ msg_ex_cnt++;
+ } while (keep_logging);
+
+ return ret;
+}
diff --git a/drivers/firmware/imx/ele_base_msg.h b/drivers/firmware/imx/ele_base_msg.h
new file mode 100644
index 000000000000..4c3699543e87
--- /dev/null
+++ b/drivers/firmware/imx/ele_base_msg.h
@@ -0,0 +1,98 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2025 NXP
+ *
+ * Header file for the EdgeLock Enclave Base API(s).
+ */
+
+#ifndef ELE_BASE_MSG_H
+#define ELE_BASE_MSG_H
+
+#include <linux/device.h>
+#include <linux/types.h>
+
+#include "se_ctrl.h"
+
+#define ELE_NONE_VAL 0x0
+
+#define ELE_GET_INFO_REQ 0xda
+#define ELE_GET_INFO_REQ_MSG_SZ 0x10
+#define ELE_GET_INFO_RSP_MSG_SZ 0x08
+
+#define MAX_UID_SIZE (16)
+#define DEV_GETINFO_ROM_PATCH_SHA_SZ (32)
+#define DEV_GETINFO_FW_SHA_SZ (32)
+#define DEV_GETINFO_OEM_SRKH_SZ (64)
+#define DEV_GETINFO_MIN_VER_MASK 0xff
+#define DEV_GETINFO_MAJ_VER_MASK 0xff00
+#define ELE_DEV_INFO_EXTRA_SZ 0x60
+
+struct dev_info {
+ u8 cmd;
+ u8 ver;
+ u16 length;
+ u16 soc_id;
+ u16 soc_rev;
+ u16 lmda_val;
+ u8 ssm_state;
+ u8 dev_atts_api_ver;
+ u8 uid[MAX_UID_SIZE];
+ u8 sha_rom_patch[DEV_GETINFO_ROM_PATCH_SHA_SZ];
+ u8 sha_fw[DEV_GETINFO_FW_SHA_SZ];
+};
+
+struct dev_addn_info {
+ u8 oem_srkh[DEV_GETINFO_OEM_SRKH_SZ];
+ u8 trng_state;
+ u8 csal_state;
+ u8 imem_state;
+ u8 reserved2;
+};
+
+struct ele_dev_info {
+ struct dev_info d_info;
+ struct dev_addn_info d_addn_info;
+};
+
+#define ELE_GET_INFO_BUFF_SZ (sizeof(struct ele_dev_info) \
+ + ELE_DEV_INFO_EXTRA_SZ)
+
+#define GET_SERIAL_NUM_FROM_UID(x, uid_word_sz) ({\
+ const u32 *__x = (const u32 *)(x); \
+ size_t __sz = (uid_word_sz); \
+ ((u64)__x[__sz - 1] << 32) | __x[0]; \
+ })
+
+#define ELE_MAX_DBG_DMP_PKT 50
+#define ELE_DEBUG_DUMP_REQ 0x21
+#define ELE_DEBUG_DUMP_REQ_SZ 0x4
+#define ELE_DEBUG_DUMP_RSP_SZ 0x5c
+
+#define ELE_PING_REQ 0x01
+#define ELE_PING_REQ_SZ 0x04
+#define ELE_PING_RSP_SZ 0x08
+
+#define ELE_SERVICE_SWAP_REQ 0xdf
+#define ELE_SERVICE_SWAP_REQ_MSG_SZ 0x18
+#define ELE_SERVICE_SWAP_RSP_MSG_SZ 0x0c
+#define ELE_IMEM_SIZE 0x10000
+#define ELE_IMEM_STATE_OK 0xca
+#define ELE_IMEM_STATE_BAD 0xfe
+#define ELE_IMEM_STATE_WORD 0x27
+#define ELE_IMEM_STATE_MASK 0x00ff0000
+#define ELE_IMEM_EXPORT 0x1
+#define ELE_IMEM_IMPORT 0x2
+
+#define ELE_FW_AUTH_REQ 0x02
+#define ELE_FW_AUTH_REQ_SZ 0x10
+#define ELE_FW_AUTH_RSP_MSG_SZ 0x08
+
+int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info);
+int ele_fetch_soc_info(struct se_if_priv *priv, void *data);
+int ele_ping(struct se_if_priv *priv);
+int ele_service_swap(struct se_if_priv *priv, dma_addr_t addr,
+ u32 addr_size, u16 flag);
+int ele_fw_authenticate(struct se_if_priv *priv, dma_addr_t contnr_addr,
+ dma_addr_t img_addr);
+int ele_debug_dump(struct se_if_priv *priv);
+#endif
diff --git a/drivers/firmware/imx/ele_common.c b/drivers/firmware/imx/ele_common.c
new file mode 100644
index 000000000000..ca8a32a82689
--- /dev/null
+++ b/drivers/firmware/imx/ele_common.c
@@ -0,0 +1,470 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2025 NXP
+ */
+
+#include "ele_base_msg.h"
+#include "ele_common.h"
+
+/*
+ * se_update_msg_chksum() - calculate and update message checksum word.
+ * @msg: message buffer.
+ * @msg_len: message length in bytes.
+ *
+ * The message length must be 4-byte aligned. The last word is treated as the
+ * checksum field and is not included in the checksum calculation.
+ *
+ * Return:
+ * 0 on success, negative errno on failure.
+ */
+int se_update_msg_chksum(u32 *msg, u32 msg_len)
+{
+ u32 nb_words;
+ u32 chksum = 0;
+ u32 i;
+
+ if (!msg)
+ return -EINVAL;
+
+ if (msg_len % SE_MSG_WORD_SZ) {
+ pr_err("Msg-len is not 4-byte aligned.\n");
+ return -EINVAL;
+ }
+
+ nb_words = msg_len / sizeof(*msg);
+ if (nb_words < 5)
+ return -EINVAL;
+
+ /* Last word is the checksum word, so skip it. */
+ nb_words--;
+
+ for (i = 0; i < nb_words; i++)
+ chksum ^= msg[i];
+
+ msg[nb_words] = chksum;
+
+ return 0;
+}
+
+int ele_msg_rcv(struct se_if_priv *priv, struct se_clbk_handle *se_clbk_hdl)
+{
+ bool wait_uninterruptible = false;
+ unsigned long remaining_jiffies;
+ unsigned long flags;
+ int ret;
+
+ remaining_jiffies = MAX_SCHEDULE_TIMEOUT;
+ do {
+ if (wait_uninterruptible)
+ ret = wait_for_completion_timeout(&se_clbk_hdl->done,
+ remaining_jiffies);
+ else
+ ret = wait_for_completion_interruptible_timeout(&se_clbk_hdl->done,
+ remaining_jiffies);
+ if (ret == -ERESTARTSYS) {
+ /*
+ * Record that a signal was observed, then continue waiting non-
+ * interruptibly until the response arrives or the timeout
+ * expires. The caller can surface the interruption to userspace
+ * after the protocol transaction is brought back to a
+ * synchronized state.
+ */
+ if (se_clbk_hdl == &priv->waiting_rsp_clbk_hdl &&
+ READ_ONCE(se_clbk_hdl->rx_msg)) {
+ WRITE_ONCE(se_clbk_hdl->signal_rcvd, true);
+ wait_uninterruptible = true;
+ continue;
+ }
+ break;
+ }
+
+ if (ret == 0) {
+ /*
+ * The response buffer belongs to the caller of ele_msg_send_rcv()
+ * and may be freed as soon as this function returns. Clear rx_msg
+ * under clbk_rx_lock so that a late se_if_rx_callback() can
+ * observe that the waiter has timed out and must not copy into
+ * the stale buffer.
+ *
+ * If the completion has not yet been signaled, mark the firmware
+ * path busy. This acts as a circuit breaker: reject new
+ * command/response transactions until the delayed response
+ * arrives and the callback closes the breaker.
+ */
+
+ spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
+ se_clbk_hdl->rx_msg = NULL;
+ if (!completion_done(&se_clbk_hdl->done))
+ atomic_set(&priv->fw_busy, 1);
+
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+ ret = -ETIMEDOUT;
+ dev_err(priv->dev,
+ "Fatal Error: SE interface: %s0, hangs indefinitely.\n",
+ get_se_if_name(priv->if_defs->se_if_type));
+ break;
+ }
+ ret = se_clbk_hdl->rx_msg_sz;
+ break;
+ } while (ret < 0);
+
+ return ret;
+}
+
+int ele_msg_send(struct se_if_priv *priv,
+ void *tx_msg,
+ int tx_msg_sz)
+{
+ struct se_msg_hdr *header = tx_msg;
+ int err;
+
+ /*
+ * Check that the size passed as argument matches the size
+ * carried in the message.
+ */
+ if (header->size << 2 != tx_msg_sz) {
+ dev_err(priv->dev,
+ "User buf hdr: 0x%x, sz mismatced with input-sz (%d != %d).",
+ *(u32 *)header, header->size << 2, tx_msg_sz);
+ return -EINVAL;
+ }
+
+ /*
+ * The i.MX MU mailbox controller copies the payload words into MU
+ * registers synchronously from its send path. It does not retain the
+ * caller-provided tx_msg pointer after mbox_send_message() returns, so
+ * the caller-owned buffer may be released after a successful send.
+ */
+ err = mbox_send_message(priv->tx_chan, tx_msg);
+ if (err < 0) {
+ dev_err(priv->dev, "Error: mbox_send_message failure.\n");
+ return err;
+ }
+
+ return tx_msg_sz;
+}
+
+static void ele_msg_send_rcv_cleanup(struct se_if_priv *priv)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags);
+ priv->waiting_rsp_clbk_hdl.rx_msg = NULL;
+ priv->waiting_rsp_clbk_hdl.rx_msg_sz = 0;
+ spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags);
+}
+
+/* API used for send/receive blocking call. */
+int ele_msg_send_rcv(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz,
+ void *rx_msg, int exp_rx_msg_sz)
+{
+ int err;
+
+ guard(mutex)(&priv->se_if_cmd_lock);
+
+ if (atomic_read(&priv->fw_busy)) {
+ dev_dbg(priv->dev, "ELE became unresponsive.\n");
+ return -EBUSY;
+ }
+ reinit_completion(&priv->waiting_rsp_clbk_hdl.done);
+ priv->waiting_rsp_clbk_hdl.rx_msg_sz = exp_rx_msg_sz;
+ priv->waiting_rsp_clbk_hdl.rx_msg = rx_msg;
+
+ err = ele_msg_send(priv, tx_msg, tx_msg_sz);
+ if (err < 0) {
+ ele_msg_send_rcv_cleanup(priv);
+ return err;
+ }
+
+ err = ele_msg_rcv(priv, &priv->waiting_rsp_clbk_hdl);
+
+ if (priv->waiting_rsp_clbk_hdl.signal_rcvd) {
+ /*
+ * A signal was received after the command was sent. ele_msg_rcv()
+ * kept waiting until the FW/kernel protocol was synchronized again.
+ * If the transaction itself completed successfully, report the
+ * deferred signal to userspace using normal syscall-restart semantics.
+ * Do not hide real firmware/protocol errors such as -ETIMEDOUT.
+ */
+ if (err > 0)
+ err = -ERESTARTSYS;
+ priv->waiting_rsp_clbk_hdl.signal_rcvd = false;
+ dev_err(priv->dev, "Err[0x%x]:Interrupted by signal.", err);
+ }
+
+ ele_msg_send_rcv_cleanup(priv);
+
+ return err;
+}
+
+static bool check_hdr_exception_for_sz(struct se_if_priv *priv,
+ struct se_msg_hdr *header)
+{
+ /*
+ * List of API(s) header that can be accepte variable length
+ * response buffer.
+ */
+ if (header->command == ELE_DEBUG_DUMP_REQ &&
+ header->ver == priv->if_defs->base_api_ver &&
+ header->size >= 2 && header->size <= (ELE_DEBUG_DUMP_RSP_SZ / 4))
+ return true;
+
+ return false;
+}
+
+/*
+ * Callback called by mailbox FW, when data is received.
+ */
+void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg)
+{
+ struct se_clbk_handle *se_clbk_hdl;
+ struct device *dev = mbox_cl->dev;
+ struct se_msg_hdr *header;
+ bool sz_mismatch = false;
+ struct se_if_priv *priv;
+ unsigned long flags;
+ u32 rx_msg_sz;
+
+ priv = dev_get_drvdata(dev);
+
+ /* The function can be called with NULL msg */
+ if (!msg) {
+ dev_err(dev, "Message is invalid\n");
+ return;
+ }
+
+ header = msg;
+ rx_msg_sz = header->size << 2;
+
+ /* Incoming command: wake up the receiver if any. */
+ if (header->tag == priv->if_defs->cmd_tag) {
+ se_clbk_hdl = &priv->cmd_receiver_clbk_hdl;
+ spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
+ if (!se_clbk_hdl->rx_msg) {
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+ dev_warn(dev, "No command receiver registered for message: %.8x\n",
+ *((u32 *)header));
+ return;
+ }
+
+ /*
+ * cmd_tag messages are delivered only to the explicitly registered
+ * command receiver. Unlike the synchronous response waiter path, the
+ * command receiver uses a dedicated long-lived buffer installed by
+ * SE_IOCTL_ENABLE_CMD_RCV and is not subject to the timeout/circuit-
+ * breaker handling used for rsp_tag messages.
+ */
+ dev_dbg(dev, "Selecting cmd receiver: for mesg header:0x%x.",
+ *(u32 *)header);
+
+ /*
+ * Pre-allocated buffer of MAX_NVM_MSG_LEN
+ * as the NVM command are initiated by FW.
+ * Size is revealed as part of this call function.
+ */
+
+ if (rx_msg_sz > MAX_NVM_MSG_LEN) {
+ /* Store the response buffer maxsize in local variable.*/
+ rx_msg_sz = MAX_NVM_MSG_LEN;
+ sz_mismatch = true;
+ }
+
+ se_clbk_hdl->rx_msg_sz = rx_msg_sz;
+ memcpy(se_clbk_hdl->rx_msg, msg, se_clbk_hdl->rx_msg_sz);
+ complete(&se_clbk_hdl->done);
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+ if (sz_mismatch)
+ dev_err(dev,
+ "CMD-RCVER NVM: hdr(0x%x) with different sz(%d != %d).\n",
+ *(u32 *)header,
+ (header->size << 2), rx_msg_sz);
+ } else if (header->tag == priv->if_defs->rsp_tag) {
+ bool exception_for_sz_mismatch = check_hdr_exception_for_sz(priv, header);
+ u32 exp_rx_msg_sz = 0;
+
+ /*
+ * waiting_rsp_clbk_hdl.rx_msg is owned by the synchronous sender in
+ * ele_msg_send_rcv(). After timeout or error, that path clears rx_msg
+ * under clbk_rx_lock before returning to its caller, which may then free
+ * the buffer. Check rx_msg under the same lock here so a delayed response
+ * can be detected and dropped instead of copying into freed memory.
+ *
+ * A late response also closes the firmware-busy circuit breaker, allowing
+ * future command/response transactions to proceed again.
+ */
+ se_clbk_hdl = &priv->waiting_rsp_clbk_hdl;
+ exp_rx_msg_sz = se_clbk_hdl->rx_msg_sz;
+ spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags);
+ if (!se_clbk_hdl->rx_msg) {
+ /* Close circuit breaker on spinlock race */
+ atomic_set(&priv->fw_busy, 0);
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+ dev_info(dev, "ELE responded (late), recovery FW available.");
+ return;
+ }
+ dev_dbg(dev, "Selecting resp waiter: for mesg header:0x%x.",
+ *(u32 *)header);
+
+ /*
+ * For rsp_tag traffic, the sender provides the expected response
+ * buffer size. If firmware returns a different size, clamp the copy
+ * length to the caller's buffer capacity before memcpy() and report the
+ * mismatch after dropping the spinlock.
+ */
+ if (rx_msg_sz != exp_rx_msg_sz) {
+ if (!exception_for_sz_mismatch)
+ sz_mismatch = true;
+
+ se_clbk_hdl->rx_msg_sz = min(rx_msg_sz, exp_rx_msg_sz);
+ }
+ memcpy(se_clbk_hdl->rx_msg, msg, se_clbk_hdl->rx_msg_sz);
+ complete(&se_clbk_hdl->done);
+ spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags);
+
+ if (sz_mismatch)
+ dev_err(dev,
+ "Rsp to CMD: hdr(0x%x) with different sz(%d != %d).\n",
+ *(u32 *)header,
+ (header->size << 2), exp_rx_msg_sz);
+ } else {
+ dev_err(dev, "Failed to select a device for message: %.8x\n",
+ *((u32 *)header));
+ }
+}
+
+int se_val_rsp_hdr_n_status(struct se_if_priv *priv, struct se_api_msg *msg,
+ u8 msg_id, u8 sz, bool is_base_api)
+{
+ struct se_msg_hdr *header = &msg->header;
+ u32 status;
+
+ if (header->tag != priv->if_defs->rsp_tag) {
+ dev_err(priv->dev, "MSG[0x%x] Hdr: Resp tag mismatch. (0x%x != 0x%x)",
+ msg_id, header->tag, priv->if_defs->rsp_tag);
+ return -EINVAL;
+ }
+
+ if (header->command != msg_id) {
+ dev_err(priv->dev, "MSG Header: Cmd id mismatch. (0x%x != 0x%x)",
+ header->command, msg_id);
+ return -EINVAL;
+ }
+
+ if ((sz % 4) || (header->size != (sz >> 2) &&
+ !check_hdr_exception_for_sz(priv, header))) {
+ dev_err(priv->dev, "MSG[0x%x] Hdr: Cmd size mismatch. (0x%x != 0x%x)",
+ msg_id, header->size, (sz >> 2));
+ return -EINVAL;
+ }
+
+ if (is_base_api && header->ver != priv->if_defs->base_api_ver) {
+ dev_err(priv->dev,
+ "MSG[0x%x] Hdr: Base API Vers mismatch. (0x%x != 0x%x)",
+ msg_id, header->ver, priv->if_defs->base_api_ver);
+ return -EINVAL;
+ } else if (!is_base_api && header->ver != priv->if_defs->fw_api_ver) {
+ dev_err(priv->dev,
+ "MSG[0x%x] Hdr: FW API Vers mismatch. (0x%x != 0x%x)",
+ msg_id, header->ver, priv->if_defs->fw_api_ver);
+ return -EINVAL;
+ }
+
+ status = RES_STATUS(msg->data[0]);
+ if (status != priv->if_defs->success_tag) {
+ dev_err(priv->dev, "Command Id[%x], Response Failure = 0x%x",
+ header->command, status);
+ return -EPERM;
+ }
+
+ return 0;
+}
+
+int se_save_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem)
+{
+ struct ele_dev_info s_info = {0};
+ int ret;
+
+ ret = ele_get_info(priv, &s_info);
+ if (ret) {
+ dev_err(priv->dev, "Failed to get info from ELE.\n");
+ return ret;
+ }
+
+ /* Check for the imem-state before continue to save imem state. */
+ if (s_info.d_addn_info.imem_state == ELE_IMEM_STATE_BAD)
+ return 0;
+
+ /*
+ * EXPORT command will save encrypted IMEM to given address,
+ * so later in resume, IMEM can be restored from the given
+ * address.
+ *
+ * Size must be at least 64 kB.
+ */
+ ret = ele_service_swap(priv, imem->daddr, ELE_IMEM_SIZE, ELE_IMEM_EXPORT);
+ if (ret < 0) {
+ dev_err(priv->dev, "Failed to export IMEM.");
+ imem->size = 0;
+ } else if (ret > ELE_IMEM_SIZE) {
+ dev_err(priv->dev, "Invalid exported IMEM size %d.", ret);
+ imem->size = 0;
+ ret = -EIO;
+ } else {
+ dev_dbg(priv->dev,
+ "Exported %d bytes of encrypted IMEM.",
+ ret);
+ imem->size = ret;
+ }
+
+ return ret > 0 ? 0 : ret;
+}
+
+int se_restore_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem)
+{
+ struct ele_dev_info s_info;
+ int ret;
+
+ /* get info from ELE */
+ ret = ele_get_info(priv, &s_info);
+ if (ret) {
+ dev_err(priv->dev, "Failed to get info from ELE.");
+ return ret;
+ }
+ imem->state = s_info.d_addn_info.imem_state;
+
+ /* Check for the imem-state and imem-size before continue to
+ * restore imem state.
+ */
+ if (s_info.d_addn_info.imem_state != ELE_IMEM_STATE_BAD || !imem->size)
+ return -EIO;
+
+ /*
+ * IMPORT command will restore IMEM from the given
+ * address, here size is the actual size returned by ELE
+ * during the export operation
+ */
+ ret = ele_service_swap(priv, imem->daddr, imem->size, ELE_IMEM_IMPORT);
+ if (ret) {
+ dev_err(priv->dev, "Failed to import IMEM");
+ return ret;
+ }
+
+ /*
+ * After importing IMEM, check if IMEM state is equal to 0xCA
+ * to ensure IMEM is fully loaded and
+ * ELE functionality can be used.
+ */
+ ret = ele_get_info(priv, &s_info);
+ if (ret) {
+ dev_err(priv->dev, "Failed to get info from ELE.");
+ return ret;
+ }
+ imem->state = s_info.d_addn_info.imem_state;
+
+ if (s_info.d_addn_info.imem_state == ELE_IMEM_STATE_OK)
+ dev_dbg(priv->dev, "Successfully restored IMEM.");
+ else
+ dev_err(priv->dev, "Failed to restore IMEM.");
+
+ return ret;
+}
diff --git a/drivers/firmware/imx/ele_common.h b/drivers/firmware/imx/ele_common.h
new file mode 100644
index 000000000000..42460d1d5dec
--- /dev/null
+++ b/drivers/firmware/imx/ele_common.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2025 NXP
+ */
+
+#ifndef __ELE_COMMON_H__
+#define __ELE_COMMON_H__
+
+#include "se_ctrl.h"
+
+#define ELE_SUCCESS_IND 0xD6
+
+#define IMX_ELE_FW_DIR "imx/ele/"
+
+int se_update_msg_chksum(u32 *msg, u32 msg_len);
+
+int ele_msg_rcv(struct se_if_priv *priv, struct se_clbk_handle *se_clbk_hdl);
+
+int ele_msg_send(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz);
+
+int ele_msg_send_rcv(struct se_if_priv *priv, void *tx_msg, int tx_msg_sz,
+ void *rx_msg, int exp_rx_msg_sz);
+
+void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg);
+
+int se_val_rsp_hdr_n_status(struct se_if_priv *priv, struct se_api_msg *msg,
+ u8 msg_id, u8 sz, bool is_base_api);
+
+/* Fill a command message header with a given command ID and length in bytes. */
+static inline void se_fill_cmd_msg_hdr(struct se_if_priv *priv, struct se_msg_hdr *hdr,
+ u8 cmd, u32 len, bool is_base_api)
+{
+ hdr->tag = priv->if_defs->cmd_tag;
+ hdr->ver = (is_base_api) ? priv->if_defs->base_api_ver : priv->if_defs->fw_api_ver;
+ hdr->command = cmd;
+ hdr->size = len >> 2;
+}
+
+int se_save_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem);
+
+int se_restore_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem);
+
+#endif /*__ELE_COMMON_H__ */
diff --git a/drivers/firmware/imx/se_ctrl.c b/drivers/firmware/imx/se_ctrl.c
new file mode 100644
index 000000000000..9bb59c200911
--- /dev/null
+++ b/drivers/firmware/imx/se_ctrl.c
@@ -0,0 +1,417 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright 2026 NXP
+ */
+
+#include <linux/bitfield.h>
+#include <linux/completion.h>
+#include <linux/delay.h>
+#include <linux/dev_printk.h>
+#include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+#include <linux/export.h>
+#include <linux/firmware.h>
+#include <linux/firmware/imx/se_api.h>
+#include <linux/genalloc.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/sys_soc.h>
+
+#include "ele_base_msg.h"
+#include "ele_common.h"
+#include "se_ctrl.h"
+
+#define MAX_SOC_INFO_DATA_SZ 256
+#define MBOX_TX_NAME "tx"
+#define MBOX_RX_NAME "rx"
+
+#define SE_TYPE_STR_DBG "dbg"
+#define SE_TYPE_STR_HSM "hsm"
+
+#define SE_TYPE_ID_DBG 0x1
+
+#define SE_TYPE_ID_HSM 0x2
+
+struct se_var_info {
+ u16 soc_rev;
+ /* To serialize populating common SoC level info. */
+ struct mutex se_var_info_lock;
+};
+
+/* contains fixed information */
+struct se_soc_info {
+ const u16 soc_id;
+ const char *soc_name;
+ const struct se_fw_img_name se_fw_img_nm;
+};
+
+struct se_if_node {
+ struct se_soc_info *se_info;
+ u8 *pool_name;
+ bool reserved_dma_ranges;
+ struct se_if_defines if_defs;
+};
+
+/* common for all the SoC. */
+static struct se_var_info var_se_info = {
+ .soc_rev = 0,
+ .se_var_info_lock = __MUTEX_INITIALIZER(var_se_info.se_var_info_lock)
+};
+
+static struct se_soc_info se_imx8ulp_info = {
+ .soc_id = SOC_ID_OF_IMX8ULP,
+ .soc_name = "i.MX8ULP",
+ .se_fw_img_nm = {
+ .prim_fw_nm_in_rfs = IMX_ELE_FW_DIR
+ "mx8ulpa2-ahab-container.img",
+ .seco_fw_nm_in_rfs = IMX_ELE_FW_DIR
+ "mx8ulpa2ext-ahab-container.img",
+ },
+};
+
+static struct se_if_node imx8ulp_se_ele_hsm = {
+ .se_info = &se_imx8ulp_info,
+ .pool_name = "sram",
+ .reserved_dma_ranges = true,
+ .if_defs = {
+ .se_if_type = SE_TYPE_ID_HSM,
+ .cmd_tag = 0x17,
+ .rsp_tag = 0xe1,
+ .success_tag = ELE_SUCCESS_IND,
+ .base_api_ver = MESSAGING_VERSION_6,
+ .fw_api_ver = MESSAGING_VERSION_7,
+ },
+};
+
+static struct se_soc_info se_imx93_info = {
+ .soc_id = SOC_ID_OF_IMX93,
+};
+
+static struct se_if_node imx93_se_ele_hsm = {
+ .se_info = &se_imx93_info,
+ .reserved_dma_ranges = true,
+ .if_defs = {
+ .se_if_type = SE_TYPE_ID_HSM,
+ .cmd_tag = 0x17,
+ .rsp_tag = 0xe1,
+ .success_tag = ELE_SUCCESS_IND,
+ .base_api_ver = MESSAGING_VERSION_6,
+ .fw_api_ver = MESSAGING_VERSION_7,
+ },
+};
+
+static const struct of_device_id se_match[] = {
+ { .compatible = "fsl,imx8ulp-se-ele-hsm", .data = &imx8ulp_se_ele_hsm },
+ { .compatible = "fsl,imx93-se-ele-hsm", .data = &imx93_se_ele_hsm },
+ { }
+};
+MODULE_DEVICE_TABLE(of, se_match);
+
+char *get_se_if_name(u8 se_if_id)
+{
+ switch (se_if_id) {
+ case SE_TYPE_ID_DBG: return SE_TYPE_STR_DBG;
+ case SE_TYPE_ID_HSM: return SE_TYPE_STR_HSM;
+ }
+
+ return NULL;
+}
+
+static struct se_fw_load_info *get_load_fw_instance(struct se_if_priv *priv)
+{
+ return &priv->load_fw;
+}
+
+static void se_soc_device_unregister(void *data)
+{
+ struct soc_device *sdev = data;
+
+ soc_device_unregister(sdev);
+}
+
+static int get_se_soc_info(struct se_if_priv *priv, const struct se_soc_info *se_info)
+{
+ struct se_fw_load_info *load_fw = get_load_fw_instance(priv);
+ struct soc_device_attribute *attr;
+ u8 data[MAX_SOC_INFO_DATA_SZ];
+ struct ele_dev_info *s_info;
+ struct soc_device *sdev;
+ int err = 0;
+
+ /*
+ * This function should be called once.
+ * Check if the se_soc_rev is zero to continue.
+ */
+ guard(mutex)(&var_se_info.se_var_info_lock);
+ if (var_se_info.soc_rev)
+ return err;
+
+ err = ele_fetch_soc_info(priv, &data);
+ if (err < 0)
+ return dev_err_probe(priv->dev, err, "Failed to fetch SoC Info.");
+ s_info = (void *)data;
+ var_se_info.soc_rev = s_info->d_info.soc_rev;
+ load_fw->imem.state = s_info->d_addn_info.imem_state;
+
+ if (!se_info->soc_name)
+ return 0;
+
+ attr = devm_kzalloc(priv->dev, sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ return -ENOMEM;
+
+ if (FIELD_GET(DEV_GETINFO_MIN_VER_MASK, var_se_info.soc_rev))
+ attr->revision = devm_kasprintf(priv->dev, GFP_KERNEL, "%x.%x",
+ FIELD_GET(DEV_GETINFO_MIN_VER_MASK,
+ var_se_info.soc_rev),
+ FIELD_GET(DEV_GETINFO_MAJ_VER_MASK,
+ var_se_info.soc_rev));
+ else
+ attr->revision = devm_kasprintf(priv->dev, GFP_KERNEL, "%x",
+ FIELD_GET(DEV_GETINFO_MAJ_VER_MASK,
+ var_se_info.soc_rev));
+
+ if (!attr->revision)
+ return -ENOMEM;
+
+ attr->soc_id = se_info->soc_name;
+
+ err = of_property_read_string(of_root, "model", &attr->machine);
+ if (err)
+ return -EINVAL;
+
+ attr->family = "Freescale i.MX";
+
+ attr->serial_number = devm_kasprintf(priv->dev,
+ GFP_KERNEL, "%016llX",
+ GET_SERIAL_NUM_FROM_UID(s_info->d_info.uid,
+ MAX_UID_SIZE >> 2));
+
+ if (!attr->serial_number)
+ return -ENOMEM;
+
+ sdev = soc_device_register(attr);
+ if (IS_ERR(sdev))
+ return PTR_ERR(sdev);
+
+ err = devm_add_action_or_reset(priv->dev, se_soc_device_unregister, sdev);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/* interface for managed res to free a mailbox channel */
+static void if_mbox_free_channel(void *mbox_chan)
+{
+ mbox_free_channel(mbox_chan);
+}
+
+static int se_if_request_channel(struct device *dev, struct mbox_chan **chan,
+ struct mbox_client *cl, const char *name)
+{
+ struct mbox_chan *t_chan;
+ int ret = 0;
+
+ t_chan = mbox_request_channel_byname(cl, name);
+ if (IS_ERR(t_chan))
+ return dev_err_probe(dev, PTR_ERR(t_chan),
+ "Failed to request %s channel.", name);
+
+ ret = devm_add_action_or_reset(dev, if_mbox_free_channel, t_chan);
+ if (ret)
+ return dev_err_probe(dev, -EPERM,
+ "Failed to add-action for removal of mbox: %s.",
+ name);
+ *chan = t_chan;
+
+ return ret;
+}
+
+static void se_if_probe_cleanup(void *plat_dev)
+{
+ struct platform_device *pdev = plat_dev;
+ struct device *dev = &pdev->dev;
+ struct se_if_priv *priv;
+
+ priv = dev_get_drvdata(dev);
+ if (!priv)
+ return;
+
+ /*
+ * In se_if_request_channel(), passed the clean-up functional
+ * pointer reference as action to devm_add_action_or_reset().
+ * No need to free the mbox channels here.
+ */
+
+ /*
+ * Being device managed buffer, no need to free the buffer allocated
+ * in se probe to store encrypted IMEM.
+ */
+
+ /*
+ * No need to check, if reserved memory is allocated
+ * before calling for its release. Or clearing the
+ * un-set bit.
+ */
+ of_reserved_mem_device_release(dev);
+
+ dev_set_drvdata(dev, NULL);
+ kfree(priv);
+}
+
+static int se_if_probe(struct platform_device *pdev)
+{
+ const struct se_soc_info *se_info;
+ const struct se_if_node *if_node;
+ struct se_fw_load_info *load_fw;
+ struct device *dev = &pdev->dev;
+ struct se_if_priv *priv;
+ int ret;
+
+ if_node = device_get_match_data(dev);
+ if (!if_node)
+ return -EINVAL;
+
+ se_info = if_node->se_info;
+
+ priv = kzalloc_obj(*priv, GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->dev = dev;
+ priv->if_defs = &if_node->if_defs;
+ dev_set_drvdata(dev, priv);
+
+ mutex_init(&priv->se_if_cmd_lock);
+ spin_lock_init(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock);
+ spin_lock_init(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock);
+ atomic_set(&priv->fw_busy, 0);
+ init_completion(&priv->waiting_rsp_clbk_hdl.done);
+ init_completion(&priv->cmd_receiver_clbk_hdl.done);
+
+ ret = devm_add_action_or_reset(dev, se_if_probe_cleanup, pdev);
+ if (ret)
+ return ret;
+
+ /* Mailbox client configuration */
+ priv->se_mb_cl.dev = dev;
+ priv->se_mb_cl.tx_block = false;
+ priv->se_mb_cl.knows_txdone = false;
+ priv->se_mb_cl.rx_callback = se_if_rx_callback;
+
+ ret = se_if_request_channel(dev, &priv->tx_chan, &priv->se_mb_cl, MBOX_TX_NAME);
+ if (ret)
+ return ret;
+
+ ret = se_if_request_channel(dev, &priv->rx_chan, &priv->se_mb_cl, MBOX_RX_NAME);
+ if (ret)
+ return ret;
+
+ if (if_node->pool_name) {
+ priv->mem_pool = of_gen_pool_get(dev->of_node, if_node->pool_name, 0);
+ if (!priv->mem_pool)
+ return dev_err_probe(dev, -ENOMEM,
+ "Unable to get sram pool = %s.",
+ if_node->pool_name);
+ }
+
+ if (if_node->reserved_dma_ranges) {
+ ret = of_reserved_mem_device_init(dev);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to init reserved memory region.");
+ }
+
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to set 32-bit coherent DMA mask.");
+
+ if (if_node->if_defs.se_if_type == SE_TYPE_ID_HSM) {
+ ret = get_se_soc_info(priv, se_info);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to fetch SoC Info.");
+ }
+
+ /* By default, there is no pending FW to be loaded.*/
+ if (se_info->se_fw_img_nm.seco_fw_nm_in_rfs) {
+ load_fw = get_load_fw_instance(priv);
+ mutex_init(&load_fw->load_fw_lock);
+
+ load_fw->se_fw_img_nm = &se_info->se_fw_img_nm;
+ load_fw->is_fw_tobe_loaded = true;
+
+ if (load_fw->se_fw_img_nm->prim_fw_nm_in_rfs) {
+ /* allocate buffer where SE store encrypted IMEM */
+ load_fw->imem.buf = dmam_alloc_coherent(priv->dev, ELE_IMEM_SIZE,
+ &load_fw->imem.daddr,
+ GFP_KERNEL);
+ if (!load_fw->imem.buf)
+ return dev_err_probe(dev, -ENOMEM,
+ "dmam-alloc-failed: To store encr-IMEM.");
+ load_fw->imem_mgmt = true;
+ }
+ }
+ dev_info(dev, "i.MX secure-enclave: %s0 interface to firmware, configured.",
+ get_se_if_name(priv->if_defs->se_if_type));
+
+ return ret;
+}
+
+static int se_suspend(struct device *dev)
+{
+ struct se_if_priv *priv = dev_get_drvdata(dev);
+ struct se_fw_load_info *load_fw;
+ int ret = 0;
+
+ load_fw = get_load_fw_instance(priv);
+
+ if (load_fw->imem_mgmt) {
+ ret = se_save_imem_state(priv, &load_fw->imem);
+ if (ret)
+ dev_warn(dev, "Failure saving IMEM state[0x%x]", ret);
+ }
+
+ return 0;
+}
+
+static int se_resume(struct device *dev)
+{
+ struct se_if_priv *priv = dev_get_drvdata(dev);
+ struct se_fw_load_info *load_fw;
+ int ret = 0;
+
+ load_fw = get_load_fw_instance(priv);
+
+ if (load_fw->imem_mgmt) {
+ ret = se_restore_imem_state(priv, &load_fw->imem);
+ if (ret)
+ dev_warn(dev, "Failure restoring IMEM state[0x%x]", ret);
+ }
+
+ return 0;
+}
+
+DEFINE_SIMPLE_DEV_PM_OPS(se_pm, se_suspend, se_resume);
+
+static struct platform_driver se_driver = {
+ .driver = {
+ .name = "fsl-se",
+ .of_match_table = se_match,
+ .pm = pm_sleep_ptr(&se_pm),
+ },
+ .probe = se_if_probe,
+};
+module_platform_driver(se_driver);
+
+MODULE_AUTHOR("Pankaj Gupta <pankaj.gupta@nxp.com>");
+MODULE_DESCRIPTION("iMX Secure Enclave Driver.");
+MODULE_LICENSE("GPL");
diff --git a/drivers/firmware/imx/se_ctrl.h b/drivers/firmware/imx/se_ctrl.h
new file mode 100644
index 000000000000..b6833abd81d0
--- /dev/null
+++ b/drivers/firmware/imx/se_ctrl.h
@@ -0,0 +1,111 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2026 NXP
+ */
+
+#ifndef SE_MU_H
+#define SE_MU_H
+
+#include <linux/bitfield.h>
+#include <linux/miscdevice.h>
+#include <linux/semaphore.h>
+#include <linux/mailbox_client.h>
+
+#define MAX_FW_LOAD_RETRIES 50
+#define SE_MSG_WORD_SZ 0x4
+
+#define RES_STATUS(x) FIELD_GET(0x000000ff, x)
+#define MAX_NVM_MSG_LEN (256)
+#define MESSAGING_VERSION_6 0x6
+#define MESSAGING_VERSION_7 0x7
+
+struct se_clbk_handle {
+ struct completion done;
+ bool signal_rcvd;
+ u32 rx_msg_sz;
+ /*
+ * Assignment of the rx_msg buffer to held till the
+ * received content as part callback function, is copied.
+ */
+ struct se_api_msg *rx_msg;
+ /*
+ * Serialise the timeout path in ele_msg_rcv() against
+ * se_if_rx_callback() so that the callback can never
+ * memcpy into a buffer that the timeout path has already
+ * freed.
+ */
+ spinlock_t clbk_rx_lock;
+};
+
+struct se_imem_buf {
+ u8 *buf;
+ dma_addr_t daddr;
+ u32 size;
+ u32 state;
+};
+
+/* Header of the messages exchange with the EdgeLock Enclave */
+struct se_msg_hdr {
+ u8 ver;
+ u8 size;
+ u8 command;
+ u8 tag;
+} __packed;
+
+#define SE_MU_HDR_SZ 4
+
+struct se_api_msg {
+ struct se_msg_hdr header;
+ u32 data[];
+};
+
+struct se_if_defines {
+ const u8 se_if_type;
+ u8 cmd_tag;
+ u8 rsp_tag;
+ u8 success_tag;
+ u8 base_api_ver;
+ u8 fw_api_ver;
+};
+
+struct se_fw_img_name {
+ const char *prim_fw_nm_in_rfs;
+ const char *seco_fw_nm_in_rfs;
+};
+
+struct se_fw_load_info {
+ const struct se_fw_img_name *se_fw_img_nm;
+ bool is_fw_tobe_loaded;
+ bool imem_mgmt;
+ struct se_imem_buf imem;
+ /* to serialize the fw load state */
+ struct mutex load_fw_lock;
+};
+
+struct se_if_priv {
+ struct device *dev;
+
+ struct se_clbk_handle cmd_receiver_clbk_hdl;
+ /*
+ * Update to the waiting_rsp_dev, to be protected
+ * under se_if_cmd_lock.
+ */
+ struct se_clbk_handle waiting_rsp_clbk_hdl;
+ /*
+ * prevent new command to be sent on the se interface while previous
+ * command is still processing. (response is awaited)
+ */
+ struct mutex se_if_cmd_lock;
+
+ struct mbox_client se_mb_cl;
+ struct mbox_chan *tx_chan, *rx_chan;
+
+ struct gen_pool *mem_pool;
+ const struct se_if_defines *if_defs;
+ struct se_fw_load_info load_fw;
+
+ atomic_t fw_busy;
+};
+
+char *get_se_if_name(u8 se_if_id);
+#endif
diff --git a/include/linux/firmware/imx/se_api.h b/include/linux/firmware/imx/se_api.h
new file mode 100644
index 000000000000..b1c4c9115d7b
--- /dev/null
+++ b/include/linux/firmware/imx/se_api.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright 2025 NXP
+ */
+
+#ifndef __SE_API_H__
+#define __SE_API_H__
+
+#include <linux/types.h>
+
+#define SOC_ID_OF_IMX8ULP 0x084d
+#define SOC_ID_OF_IMX93 0x9300
+
+#endif /* __SE_API_H__ */
--
2.43.0
^ permalink raw reply related
* [PATCH v28 2/7] dt-bindings: arm: fsl: add imx-se-fw binding doc
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
The NXP security hardware IP(s) like: i.MX EdgeLock Enclave, V2X etc.,
creates an embedded secure enclave within the SoC boundary to enable
features like:
- HSM
- SHE
- V2X
Secure-Enclave(s) communication interface are typically via message
unit, i.e., based on mailbox linux kernel driver. This driver enables
communication ensuring well defined message sequence protocol between
Application Core and enclave's firmware.
Driver configures multiple misc-device on the MU, for multiple
user-space applications, to be able to communicate over single MU.
It exists on some i.MX processors. e.g. i.MX8ULP, i.MX93 etc.
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
.../devicetree/bindings/firmware/fsl,imx-se.yaml | 91 ++++++++++++++++++++++
1 file changed, 91 insertions(+)
diff --git a/Documentation/devicetree/bindings/firmware/fsl,imx-se.yaml b/Documentation/devicetree/bindings/firmware/fsl,imx-se.yaml
new file mode 100644
index 000000000000..fa81adbf9b80
--- /dev/null
+++ b/Documentation/devicetree/bindings/firmware/fsl,imx-se.yaml
@@ -0,0 +1,91 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/firmware/fsl,imx-se.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NXP i.MX HW Secure Enclave(s) EdgeLock Enclave
+
+maintainers:
+ - Pankaj Gupta <pankaj.gupta@nxp.com>
+
+description: |
+ NXP's SoC may contain one or multiple embedded secure-enclave HW
+ IP(s) like i.MX EdgeLock Enclave, V2X etc. These NXP's HW IP(s)
+ enables features like
+ - Hardware Security Module (HSM),
+ - Security Hardware Extension (SHE), and
+ - Vehicular to Anything (V2X)
+
+ Communication interface to the secure-enclaves(se) is based on the
+ messaging unit(s).
+
+properties:
+ compatible:
+ enum:
+ - fsl,imx8ulp-se-ele-hsm
+ - fsl,imx93-se-ele-hsm
+ - fsl,imx95-se-ele-hsm
+
+ mboxes:
+ items:
+ - description: mailbox phandle to send message to se firmware
+ - description: mailbox phandle to receive message from se firmware
+
+ mbox-names:
+ items:
+ - const: tx
+ - const: rx
+
+ memory-region:
+ maxItems: 1
+
+ sram:
+ maxItems: 1
+
+required:
+ - compatible
+ - mboxes
+ - mbox-names
+
+allOf:
+ # memory-region
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx8ulp-se-ele-hsm
+ - fsl,imx93-se-ele-hsm
+ then:
+ required:
+ - memory-region
+ else:
+ properties:
+ memory-region: false
+
+ # sram
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - fsl,imx8ulp-se-ele-hsm
+ then:
+ required:
+ - sram
+
+ else:
+ properties:
+ sram: false
+
+additionalProperties: false
+
+examples:
+ - |
+ secure-enclave {
+ compatible = "fsl,imx95-se-ele-hsm";
+ mboxes = <&ele_mu0 0 0>, <&ele_mu0 1 0>;
+ mbox-names = "tx", "rx";
+ };
+...
--
2.43.0
^ permalink raw reply related
* [PATCH v28 1/7] Documentation/firmware: add imx/se to other_interfaces
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel
In-Reply-To: <20260717-imx-se-if-v28-0-0a9659c7e69d@nxp.com>
From: Pankaj Gupta <pankaj.gupta@nxp.com>
Documents i.MX SoC's Service layer and C_DEV driver for selected SoC(s)
that contains the NXP hardware IP(s) for Secure Enclaves(se) like:
- NXP EdgeLock Enclave on i.MX93 & i.MX8ULP
Signed-off-by: Pankaj Gupta <pankaj.gupta@nxp.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
.../driver-api/firmware/other_interfaces.rst | 133 +++++++++++++++++++++
1 file changed, 133 insertions(+)
diff --git a/Documentation/driver-api/firmware/other_interfaces.rst b/Documentation/driver-api/firmware/other_interfaces.rst
index 06ac89adaafb..6c6fa9a0ba1d 100644
--- a/Documentation/driver-api/firmware/other_interfaces.rst
+++ b/Documentation/driver-api/firmware/other_interfaces.rst
@@ -49,3 +49,136 @@ of the requests on to a secure monitor (EL3).
.. kernel-doc:: drivers/firmware/stratix10-svc.c
:export:
+
+NXP Secure Enclave Firmware Interface
+=====================================
+
+Introduction
+------------
+The NXP's i.MX HW IP like EdgeLock Enclave, V2X etc., creates an embedded secure
+enclave within the SoC boundary to enable features like:
+
+- Hardware Security Module (HSM)
+- Security Hardware Extension (SHE)
+- Vehicular to Anything (V2X)
+
+Each of the above features is enabled through dedicated NXP H/W IP on the SoC.
+On a single SoC, multiple hardware IP (or can say more than one secure enclave)
+can exist.
+
+NXP SoCs enabled with the such secure enclaves(SEs) IPs are:
+i.MX93, i.MX8ULP
+
+To communicate with one or more co-existing SE(s) on SoC, there is/are dedicated
+messaging units(MU) per SE. Each co-existing SE can have one or multiple exclusive
+MUs, dedicated to itself. None of the MU is shared between two SEs. Communication
+of the MU is realized using the mailbox driver. Each secure enclave can cater to
+multiple clients by virtue of these exclusive MUs. Also, they can distinguish
+transactions originating from these clients based on the MU used and core security
+state. The communication between the clients and secure enclaves is in the form of
+a command/response mechanism. Each client could expose a specific set of secure enclave
+features to the higher layers, based on the commands supported by that client. For
+example, the secure enclave could simultaneously support an OPTEE TA and Linux
+middleware as clients. Each of these clients can expose a specific set of secure
+enclave features based on the command set supported by them.
+
+NXP Secure Enclave(SE) Interface
+--------------------------------
+MU(s) is/are not shared between SE(s). But for an SoC like i.MX95 which has
+multiple SE(s) like HSM, V2X-HSM, V2X-SHE, all the SE(s) and their interfaces 'se-if'
+that is/are dedicated to a particular SE will be enumerated and provisioned using the
+single compatible node("fsl,imx95-se").
+
+Each 'se-if' comprises two layers:
+
+- (C_DEV Layer) User-Space software-access interface.
+- (Service Layer) OS-level software-access interface.
+
+::
+
+ +--------------------------------------------+
+ | Character Device(C_DEV) |
+ | |
+ | +---------+ +---------+ +---------+ |
+ | | misc #1 | | misc #2 | ... | misc #n | |
+ | | dev | | dev | | dev | |
+ | +---------+ +---------+ +---------+ |
+ | +-------------------------+ |
+ | | Misc. Dev Synchr. Logic | |
+ | +-------------------------+ |
+ | |
+ +--------------------------------------------+
+
+ +--------------------------------------------+
+ | Service Layer |
+ | |
+ | +-----------------------------+ |
+ | | Message Serialization Logic | |
+ | +-----------------------------+ |
+ | +---------------+ |
+ | | imx-mailbox | |
+ | | mailbox.c | |
+ | +---------------+ |
+ | |
+ +--------------------------------------------+
+
+- service layer:
+ This layer is responsible for ensuring the communication protocol that is defined
+ for communication with firmware.
+
+ FW Communication protocol ensures two things:
+
+ - Serializing the messages to be sent over an MU.
+ - FW can handle one command message at a time.
+
+- c_dev:
+ This layer offers character device contexts, created as '/dev/<se>_mux_chx'.
+ Using these multiple device contexts that are multiplexed over a single MU,
+ userspace application(s) can call fops like write/read to send the command message,
+ and read back the command response message to/from Firmware.
+ fops like read & write use the above defined service layer API(s) to communicate with
+ Firmware.
+
+ Misc-device(/dev/<se>_mux_chn) synchronization protocol::
+
+ Non-Secure + Secure
+ |
+ |
+ +-----------+ +-------------+ |
+ | se_ctrl.c +<---->+imx-mailbox.c| |
+ | | | mailbox.c +<-->+------+ +------+
+ +-----+-----+ +-------------+ | MU X +<-->+ ELE |
+ | +------+ +------+
+ +----------------+ |
+ | | |
+ v v |
+ logical logical |
+ receiver waiter |
+ + + |
+ | | |
+ | | |
+ | +----+------+ |
+ | | | |
+ | | | |
+ device_ctx device_ctx device_ctx |
+ |
+ User 0 User 1 User Y |
+ +------+ +------+ +------+ |
+ |misc.c| |misc.c| |misc.c| |
+ kernel space +------+ +------+ +------+ |
+ |
+ +---------------------------------------------------- |
+ | | | |
+ userspace /dev/ele_muXch0 | | |
+ /dev/ele_muXch1 | |
+ /dev/ele_muXchY |
+ |
+
+When a user sends a command to the firmware, it registers its device_ctx
+as waiter of a response from firmware.
+
+Enclave's Firmware owns the storage management over a Linux filesystem.
+For this c_dev provisions a dedicated slave device called "receiver".
+
+.. kernel-doc:: drivers/firmware/imx/se_ctrl.c
+ :export:
--
2.43.0
^ permalink raw reply related
* [PATCH v28 0/7] firmware: imx: driver for NXP secure-enclave
From: pankaj.gupta @ 2026-07-17 11:45 UTC (permalink / raw)
To: Jonathan Corbet, Shuah Khan, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Frank Li, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Pankaj Gupta
Cc: linux-doc, linux-kernel, devicetree, imx, linux-arm-kernel,
Frieder Schrempf, sashiko-bot
The NXP's i.MX EdgeLock Enclave, a HW IP creating an embedded secure
enclave within the SoC boundary to enable features like
- HSM
- SHE
- V2X
Communicates via message unit with linux kernel. This driver is
enables communication ensuring well defined message sequence protocol
between Application Core and enclave's firmware.
Driver configures multiple misc-device on the MU, for multiple
user-space applications can communicate on single MU.
It exists on some i.MX processors. e.g. i.MX8ULP, i.MX93 etc.
-------
Changes in v28:
5/7: Fix 10 of the 13 issues reported by Sashiko AI review
- [High] se_ctrl.c: se_if_fops_read(): reacquire fops_lock after
ele_msg_rcv() returns before accessing pending lists or rx_msg.
fops_lock is dropped before the blocking wait; a concurrent close could
free the DMA buffers and pending lists while the read is blocked, leading
to UAF and list corruption. Re-check cleanup_done under fops_lock before
touching any shared state.
- [High] ele_common.c: se_val_rsp_hdr_n_status(): guard msg->data[0] read
with if (header->size > SE_MU_HDR_WORD_SZ). A header-only response (1
word) is valid; the unconditional read caused a KASAN slab-out-of-bounds.
- [High] ele_common.h: reduce SE_RCV_MSG_DEFAULT_TIMEOUT from 5000 s to
3000 ms. After a signal interrupts the interruptible wait, ele_msg_rcv()
switches to TASK_UNINTERRUPTIBLE. A 5000-second uninterruptible sleep
reliably triggers the hung-task watchdog. 3000 ms is well below the
default 120 s threshold.
- [High] se_ctrl.c: cleanup_se_shared_mem(): call
se_dev_ctx_shared_mem_cleanup() to free se_buf_desc list entries before
releasing the DMA backing memory, fixing a leak when the fd is closed
with pending I/O buffers.
- [High] se_ctrl.c: se_dev_ctx_shared_mem_cleanup(): skip memset of DMA
buffers when the context is the fw_busy one (command timed out). The
firmware may still be actively accessing the buffer; zeroing it would
corrupt the in-flight DMA transaction.
- [High] se_ctrl.c: cleanup_se_shared_mem(): guard against calling
dma_free_coherent() with a NULL ptr (probe failure before DMA alloc
succeeded).
- [High] The command-receiver rx_msg_sz update is now done under
cmd_receiver_clbk_hdl.clbk_rx_lock. The read path also validates
cmd_receiver_clbk_hdl.dev_ctx, rx_msg, and rx_msg_sz under the same callback
lock before consuming the message.
- [High] se_ctrl.c: se_if_probe_cleanup(): call se_clear_fw_busy() before
cancel_work_sync(). A late mailbox interrupt arriving between the two
calls would see fw_busy still set and schedule work on the already-
cancelled fw_busy_work, causing a use-after-free.
- [Medium] se_ctrl.c: cleanup_dev_ctx(): remove goto from inside a
scoped_guard() block. Use a local already_done flag instead to keep the
cleanup path flat and avoid confusing ownership semantics.
- [Misc] se_ctrl.c: se_ctrl.h: add SE_MU_HDR_WORD_SZ = 1 constant.
Make se_is_fw_busy_ctx(), se_dev_ctx_shared_mem_cleanup() and
get_shared_mem_slot() non-static so they can be called from ele_common.c
and future callers.
4/7: Fix init_misc_device_context() to return 0 on the success path.
- The helper initialized ret to -ENOMEM and returned ret even after
successfully allocating and initializing priv_dev_ctx. This caused
se_if_probe() to treat a successful init_misc_device_context() call as a
failure and abort probe. Return 0 after assigning *new_dev_ctx.
- Avoid a possible NULL pointer dereference in se_if_rx_callback() after
waking the synchronous response waiter. The response callback used
se_clbk_hdl->dev_ctx->devname after calling complete() and dropping
clbk_rx_lock. The awakened ele_msg_send_rcv() cleanup path can clear
waiting_rsp_clbk_hdl.dev_ctx under the same lock before the size-mismatch
dev_err() is emitted. Snapshot devname while clbk_rx_lock is still held and
use the local copy after dropping the lock.
3/7: Fix several issues reported by Sashiko in the ELE driver:
- Add cleanup helper for ele_get_info() and remove the goto-based cleanup path
that mixed manual cleanup with scoped __free(kfree) objects.
- Document the mailbox TX buffer lifetime assumption in ele_msg_send().
The i.MX MU mailbox controller copies message payload words into MU
registers synchronously and does not retain the caller-provided tx_msg
pointer after mbox_send_message() returns.
- Replace se_get_msg_chksum() with se_update_msg_chksum(), which validates
the message pointer and size, calculates the checksum, and updates the
checksum word directly.
- Remove the stale ret check after se_fill_cmd_msg_hdr(), since
se_fill_cmd_msg_hdr() now returns void.
- Validate the encrypted IMEM export size returned by firmware against
ELE_IMEM_SIZE before storing it in imem->size for later resume-time import.
- Add a mutex to serialize population of common SoC-level information stored
in var_se_info.
- Add NULL checks for devm_kasprintf() results before passing revision and
serial-number strings to soc_device_register().
- Set a 32-bit DMA/coherent mask to make the ELE 32-bit firmware address
constraint explicit, while still relying on the reserved memory region for
ELE-accessible DMA memory.
- Keep mutable firmware-load and IMEM state per device in struct se_if_priv,
while var_se_info only caches immutable SoC-level revision data.
- Keep the SoC device unregister devres action and devm-managed IMEM coherent
memory handling from v27.
Reference:
- Link to v27: https://lore.kernel.org/r/20260715-imx-se-if-v27-0-bb7c45952f06@nxp.com
Changes in v27:
Address Sashiko findings around:
Changes in v27:
Address Sashiko findings around:
- Encrypted IMEM DMA address handling, response waiter cleanup,
service-swap address validation, SoC device unregister,
devm-managed IMEM cleanup, timeout classification, signal
handling, firmware-load serialization, UAPI compatibility, and
timed-out firmware transaction cleanup.
- Use dma_addr_t for ELE firmware authentication and IMEM service-swap
buffers, store the encrypted IMEM DMA handle in the IMEM state, and pass
imem->daddr to IMEM save/restore.
- Reject ELE service-swap addresses whose upper 32 bits are set before
placing the address into the 32-bit firmware message field.
- Clear waiting_rsp_clbk_hdl state under clbk_rx_lock on all send/receive
exit paths, including ele_msg_send() failure.
- Keep mailbox TX completion handled by the mailbox controller/core with
knows_txdone set to false. The i.MX MU controller copies the message
payload into hardware registers synchronously.
- Move mutable firmware-load and IMEM state to per-device storage in
struct se_if_priv.
- Register a devres cleanup action to unregister the SoC device returned by
soc_device_register().
- Remove manual freeing of the devm-managed encrypted IMEM coherent buffer
from probe cleanup.
- Fix response-wait timeout classification by basing timeout handling on
the callback handle being waited on, not on command-receiver file context
identity.
- Use fixed-width UAPI fields and u64_to_user_ptr() to support 32-bit
userspace compatibility.
- Add compat_ioctl support for the misc-device UAPI.
- Bound userspace-provided message sizes before memdup_user().
- Track timed-out firmware transactions with the corresponding dev_ctx so
coherent DMA memory is not freed while firmware may still access it.
- Serialize firmware loading state with load_fw_lock to avoid concurrent
firmware authentication requests.
- Fix get_se_soc_id() to use the correct match-data type.
- Fix command-receiver lifecycle handling by clearing command-receiver
callback state under clbk_rx_lock and freeing rx_msg only after dropping
the lock.
- Avoid unbind/remove deadlock by not holding dev_ctx->fops_lock across the
blocking command-receiver read wait.
- Initialize dev_ctx reference counting with kref_init().
- Add ELE FW API command filtering through ele_uapi_allowed_fw_cmd().
- Move FW API command IDs and FW API-specific state transitions into
ele_fw_api.c/ele_fw_api.h.
- Allow command-receiver registration/unregistration through the supported
storage open/close FW API flow.
- Refactor shared-memory slot allocation and rollback in
se_ioctl_setup_iobuf_handler().
- Fix hsm0 device tree indentation reported by Sashiko.
- Address Lothar Waßmann's review comments:
- make se_fill_cmd_msg_hdr() return void and remove dead error checks;
- remove the trailing comma after the final empty of_device_id sentinel.
- Validate the series with checkpatch, sparse, W=1/W=2 builds, coccicheck,
dt_binding_check for fsl,imx-se.yaml, CHECK_DTBS=y for imx8ulp-evk.dtb,
and headers_install for include/uapi/linux/se_ioctl.h.
Reference:
- Link to v26: https://lore.kernel.org/r/20260629-imx-se-if-v26-0-146446285744@nxp.com
Changes in v26:
- Folded kernel test robot and Sashiko-bot fixes into the series.
- Added MAILBOX dependency for COMPILE_TEST builds.
- Hardened response waiter timeout handling and late-response processing.
- Serialized command receiver registration and callback-visible receiver state.
- Added iobuf round_up() overflow detection.
- Rolled back iobuf shared-memory reservation on setup failures.
- Bounded userspace-controlled response buffer size.
- Preserved ioctl operation errors across cleanup.
- Added explicit priv/dev_ctx lifetime handling and teardown/open serialization.
Testing:
- checkpatch.pl --strict: no warnings
- sparse: no warnings
- coccicheck on drivers/firmware/imx/: no driver-specific warnings
Reference:
- Link to v25: https://lore.kernel.org/r/20260122-imx-se-if-v25-0-5c3e3e3b69a8@nxp.com
Changes in v25:
5/7
- removes kernel bot reported warning errors.
3/7
- fix checkpatch --strict error.
1/2, 2/7, 4/7, 6/7 & 7/7
- No changes
Reference:
- Link to v24: https://lore.kernel.org/r/20260121-imx-se-if-v24-0-c5222df51cc2@nxp.com
Changes in v24:
5/7 & 3/7
- removes kernel bot reported warning errors.
1/2, 2/7, 4/7, 6/7 & 7/7
- No changes
Reference:
- Link to v23: https://lore.kernel.org/r/20251219-imx-se-if-v23-0-5c6773d00318@nxp.com
Changes in v23:
5/7
- removed un-neccessary 'kfree' from the func se_ioctl_cmd_snd_rcv_rsp_handler().
1/2, 2/7, 3/7, 4/7, 6/7 & 7/7
- No changes
Reference:
- Link to v22: https://lore.kernel.org/r/20251218-imx-se-if-v22-0-07418c872509@nxp.com
Changes in v22:
3/7 & 5/7
- reverted to previous change of using "__free(kfree)", by declare-and-initialize __free() vars next to their allocations.
1/7
- rename the se_fw.c to se_ctrl.c
2/7, 4/7, 5/7 & 7/7
- No changes.
Reference:
- Link to v21: https://lore.kernel.org/r/20251212-imx-se-if-v21-0-ee7d6052d848@nxp.com
Changes in v21:
3/7
- smatch warning fixes.
- Added "COMPILE_TEST" into "depends on IMX_MBOX && ARCH_MXC && ARM64"
- removed "__free(kfree)" & added kfree();
5/7
- removed "__free(kfree)" & added kfree();
1/2, 2/7, 4/7, & 7/7
- No changes
Reference:
- Link to v20: https://lore.kernel.org/r/20251203-imx-se-if-v20-0-a04a25c4255f@nxp.com
Changes in v20:
5/7:
- adds a func "se_chk_tx_msg_hdr", to check the validity of the in-coming message from usersapce.
1/2, 2/7, 3/7, 4/7, 6/7 & 7/7
- No changes
Reference:
- Link to v19: https://lore.kernel.org/r/20250927-imx-se-if-v19-0-d1e7e960c118@nxp.com
Changes in v19:
1/7
- Added 9 lines to the Introduction from line 73-82.
3/7
- Update the commit message for " For i.MX9x SoC(s) there is at least one
dedicated ELE MU(s) for each world - Linux(one or more) and OP-TEE OS (one or
more), that needs to be shared between them.."
Reference:
- Link to v18: https://lore.kernel.org/r/20250619-imx-se-if-v18-0-c98391ba446d@nxp.com
Changes in v18:
1/7
- Wrap both diagrams above in literal code block by using double-colon
3/7 & 5/7
- Collected Frank's R-b tag.
2/7, 4/7, 6/7 & 7/7
- No changes
Reference:
- Link to v17: https://lore.kernel.org/r/20250426-imx-se-if-v17-0-0c85155a50d1@nxp.com
Changes in v17:
- Changes to 3/7 & 5/7: to wrap code text at 80 character whereever possible.
Reference:
- Link to v16: https://lore.kernel.org/r/20250409-imx-se-if-v16-0-5394e5f3417e@nxp.com
Changes in v16:
- commit 3/7 and 4/7 are moved to end commits making them as 6/7 and 7/7 respectively.
- No change in 1/7 & 2/7.
7/7
- Collected Frank's R-b tag.
6/7
- commit message is updated to wrap at 75 characters.
5/7
- func add_b_desc_to_pending_list, removed the initialization of b_desc to
NULL.
- variable timeout in func ele_msg_rcv(), is renamed to timeout_ms.
- struct se_if_priv, member variable se_rcv_msg_timeout, is renamed to
se_rcv_msg_timeout_ms.
- in func load_firmware, move the label exit after dma_free_coherent.
4/7
- commit message is updated to wrap at 75 characters.
3/7
- ele_debug_dump, updated the assignment of keep_logging.
- ele_fw_authenticate function definition is updated to take two address
as arguments.
Reference:
- Link to v15: https://lore.kernel.org/r/20250407-imx-se-if-v15-0-e3382cecda01@nxp.com
Changes in v15:
- Patch 3/6 is split into two:
- 3/7: arm64: dts: imx8ulp-evk: add reserved memory property
- 4/7: arm64: dts: imx8ulp: add nxp secure enclave firmware
- No change in 1/7 & 2/7.
7/7
- removed the se_intance_id structure member variable.
- replace variable name from wait to timeout.
- used 'goto' to follow the common exit path calling "release_firmware(fw);" in case of error path.
- removed TBD string.
- Used ARRAY_SIZE(pending_lists).
- moved init_device_context after init_misc_device_context.
- defined err as long to avoid force convert in func
- added se_rcv_msg_timeout to priv, to control probe/suspend/resume per interface.
6/7
- removed the se_intance_id structure member variable.
- Added dev_ctx to the structure se_clbk_handle, too.
- Collected Frank's R-b tag.
5/7
- removed the se_intance_id structure member variable.
- since added se_if_probe_cleanup to devm, se_if_remove() is redundant. hence removed it.
- rename se_add_msg_chksum to se_get_msg_chksum
- added check if msg-size is 4 byte aligned.
- Fixed multiline comments.
- ele_debug_dump api is updated as part of comment disposition like single setting of flag "keep_logging" & adding if (ret).
- moved dev_err to dev_dbg, for imem save/restore functions.
- moved func get_se_if_name, from 7/7 to here.
3/7
- Updated the commit message.
- split the current patch into two:
-- 3/7 for board dts, and
-- 4/7 for chip dts
Reference:
- Link to v14: https://lore.kernel.org/r/20250327-imx-se-if-v14-0-2219448932e4@nxp.com
Changes in v14:
- Patch 5/5 is split into two:
- firmware: drivers: imx: adds miscdev
- Introduce dev-ctx dedicated to private.
-- Base patch before enabling misc-device context, to have the send-receive path, based on device context.
- No change in 1/6 & 2/6.
- Copied change logs from individual commits.
6/6
- moved definition of func se_load_firmware, from 4/6 patch to this patch.
- split init_device_context to init_misc_device_context.
- Different value of se_rcv_msg_timeout is required to be set. Receiving the response of 4K RSA operation can to take upto 3 minutes.
This long value cannot be set during Linux: boot-up and suspend-resume.
Hence, it will be set to default small-value during Linux: boot-up and suspend-resume.
- func se_dev_ctx_cpy_out_data(), in either case: do_cpy true or false, the clean-up needs to be done and it is implemented like wise.
Once do_cpy is false, no need to continue copy to user buffer. But continue to do clean-up. hence cannot return.
And every dev-ctx operation is done after taking the lock. Hence, two operations with same dev-ctx is not possible in parallel.
- func "init_device_context", for 0th misc dev_ctx, which is created at the time of probe, the device memory management is required. hence there is a difference.
- func "init_device_context", dev_er is replaced with return dev_err_probe.
- func "init_device_context", devm_add_action is replaced by devm_add_action_reset.
- removed type-cast from func se_ioctl_get_se_soc_info_handler().
- used scoped_cond_guard(mutex, _intr, return -EBUSY, &<mutex_lock>)
- combined dev_err & dev_dbg to one dev_err in se_if_fops_read().
- removed the structure member "se_shared_mem_mgmt->secure_mem".
4/6
- trimmed the ele_fetch_soc_info.
- removed the function ptr "se_info->se_fetch_soc_info" and replaced with ele_fetch_soc_info.
- moved definition of func se_load_firmware, to 6/6 patch.
- Different SoC, different ways to fetch soc_info. Generic function declaration for ele_fetch_soc_info() is needed. Hence wrapping ele_get_info() in it.
- Updated Kconfig help text for assertive tone.
- func ele_debug_dump is updated, to remove constructing the format string.
- removed the macro usage for SOC_ID_MASK.
- used low case hex number.
- Condition will never occur, where msg_len satisfy the following condition "msg_len % 4 != 0". Err msg is added if it occurs.
- Function description is added to se_add_msg_crc.
- timeout is added to function ele_msg_rcv, in 5/5 patch.
- local variable "header" is initialized with "tx_msg" and replaced "return err" with "return tx_msg_sz" in func ele_msg_send().
- replace function name from "exception_for_size" to "check_hdr_exception_for_sz"
- replaced "return ret > 0 ? 0 : -1;" with "return ret > 0 ? 0 : ret;" in func "se_save_imem_state".
- func "se_restore_imem_state", to return if the condition is false to proceed.
- removed casting by (void *).
- removed devm_kasprintf and done direct allocatiion for attr->soc_id = "i.MX8ULP" & attr->soc_id = "i.MX8ULP", & attr->family.
- Followed Reverse christmas tree order, whereever missing.
- There is no return if ele_fw_authenticate fails. Execution flow continue forward and execute the fucn dma_free_coherent().
- The loop is not for retry. The loop is needed to load secondary fw followed by loading primary fw, first. This is the case when ELE also got reset.
- dev_err_probe is corrected in func "se_if_request_channel".
3/6
-
Reference:
- Link to v13: https://lore.kernel.org/r/20250311-imx-se-if-v13-0-9cc6d8fd6d1c@nxp.com
Changes in v13:
5/5
- Updated the commit message for imperative mood.
- Remove the usage of macros- NODE_NAME, GET_ASCII_TO_U8, GET_IDX_FROM_DEV_NODE_NAME.
- Clean-up the return path by replacing "ret = -<err>; return ret;" with "return -<err>;"
- Clean-up the return path by replacing "ret = -<err>; goto exit;" with "return -<err>;"
- Removed goto statements from the entire driver, where there is no common code at function's exit.
- Fixes the check-patch erros reported with flag "--strict"
- Replaced devm_add_action, with devm_add_action_or_reset
- Removed the un-necesary and obvious code comments.
- Removed dev_probe_err at the exit of function se_if_probe().
4/5
- Clean-up the return path by replacing "ret = -<err>; return ret;" with "return -<err>;"
- Clean-up the return path by replacing "ret = -<err>; goto exit;" with "return -<err>;"
- Removed goto statements from the entire driver, where there is no common code at function's exit.
- fixes the check-patch erros reported with flag "--strict"
- removed the un-necesary and obvious code comments.
- variable received msg timeout to be different at boot-up & suspend/resume and send/recv ioctlis.
3/5
- compatible string is modified from "fsl,imx8ulp-se" to "fsl,imx8ulp-se-ele-hsm".
- updated the alias name.
2/5
- compatible string is modified from "fsl,imx8ulp-se" to "fsl,imx8ulp-se-ele-hsm".
- compatible string is modified from "fsl,imx93-se" to "fsl,imx93-se-ele-hsm".
- compatible string is modified from "fsl,imx95-se" to "fsl,imx95-se-ele-hsm".
- Mis-understood the +1 from Conor. Hence dropped the Reviewed-by tag.
- Collected Rob's R-b tag on v7 (https://lore.kernel.org/all/172589152997.4184616.5889493628960272898.robh@kernel.org/)
1/5
- No change
Reference:
- Link to v12: https://lore.kernel.org/r/20250120-imx-se-if-v12-0-c5ec9754570c@nxp.com
Changes in v12:
5/5
- increased the wait-timeout.
4/5
- rename flag "handle_susp_resm" to "imem_mgmt"
- moved the buffer allocation ot load_fw->imem.buf, to se_probe_if.
- setting imem state at initialization.
3/5
- No change
2/5
- No change
1/5
- No change
Reference:
- Link to v11: https://lore.kernel.org/r/20241220-imx-se-if-v11-0-0c7e65d7ae7b@nxp.com
Changes in v11:
5/5
- devname is constructed by concatinating get_se_if_name(se_if_id) & se_if_instance_id.
- ele_rcv_msg(), is updated to add the wait_interruptible_timeout for the non-NVM-Daemon message exchanges, such that in case of no response from FW,
Linux donot hangs.
- added a new helper function get_se_if_name(), to return the secure-enclave interface owner's name string.
- added a new helper function get_se_soc_id(), to return the secure-enclave's SoC id.
4/5
- moved the se_if_node_info member "soc_register", to the struct "se_if_node_info_list"; as soc registration done once, not per interface.
- moved the se_if_node_info member "se_fetch_soc_info", to the struct "se_if_node_info_list"; as soc info fetching is done once, not per interface.
- Added two member variable se_if_id and se_if_instance_id to struct se_if_defines.
- removed the member "se_name" from struct "se_if_node_info". Rather, it will constructed by concatinating get_se_if_name(se_if_id) & se_if_instance_id.
- moved the static global variable "se_version", to the newly created structure "struct se_var_info".
- moved the member "struct se_fw_load_info load_fw" of "se_if_node_info_list", to the newly created structure "struct se_var_info".
- Replaced RUNTIME_PM_OPS with SET_SYSTEM_SLEEP_PM_OPS, in power-managment ops.
3/5
- No change
2/5
- No change
1/5
- No change
Reference:
- Link to v10: https://lore.kernel.org/r/20241104-imx-se-if-v10-0-bf06083cc97f@nxp.com
v10: firmware: imx: driver for NXP secure-enclave
Changes in v10:
5/5
- replaced the u8, u16, u32, u64, with __u8, __u16, __u32, __u64 in
'include/uapi/linux/se_ioctl.h'.
4/5
- No change
3/5
- No change
2/5
- No change
1/5
- No change
Reference:
- Link to v9: https://lore.kernel.org/r/20241016-imx-se-if-v9-0-fd8fa0c04eab@nxp.com
Changes in v9:
4/5
- change se_if_remove function signature, required after rebase to v6.12-rc1.
- move the info->macros to a structure "struct se_if_defines if_defs".
- Removed "info" from "struct se_if_defines if_defs".
- Moved "mem_pool" from "struct se_if_defines if_defs" to "priv".
- Fetching "info" using container-of.
5/5
- Fetching "info" using container-of.
- Fixed issue reported by sparse.
Reference:
- Link to v8: https://lore.kernel.org/r/20241015-imx-se-if-v8-0-915438e267d3@nxp.com
Changes in v8:
5/5
- Remove the check for SE_IF_CTX_OPENED.
- replaced dev_ctx->priv-dev, priv->dev, whereever possible.
- func "if_misc_deregister" moved before func "init_device_context".
- func "init_device_context" before func "se_ioctl_cmd_snd_rcv_rsp_handler".
- func "se_if_fops_write" and "se_if_fops_read", are moved after func "se_ioctl_get_mu_info".
- non static functions "se_dev_ctx_cpy_out_data, se_dev_ctx_shared_mem_cleanup & init_device_context" are moved static and local scope.
- Removed back & forth between the two structs "struct se_if_device_ctx *dev_ctx" and "struct se_shared_mem_mgmt_info *se_shared_mem_mgmt"
- removed the NULL check for bdesc.
- fops_open, is corrected for acquiring the fops_lock.
- Fops_close, mutex unlock is removed. Infact check for waiting_rsp_clbk_hdl.dev_ctx, is removed.
- sema_init(&dev_ctx->fops_lock, 1);, replaced with Mutex.
- structure member se_notify, is removed.
4/5
- removed initializing err to zero in func ele_fetch_soc_info(),
- replaced 'return 0', with 'goto exit', if the condition (!priv->mem_pool) is true.
- replaced "struct *dev" with "struct se_if_priv *priv", in base_message API(s) and others.
- Created a separate structure "struct se_if_defines" to maintain interface's fixed values like cmd_tag, rsp_tag, success_tag etc.
- removed the macros "WORD_SZ", "SOC_VER_MASK", "DEFAULT_IMX_SOC_VER", "RESERVED_DMA_POOL".
- Added handling for "ctrl+c", by postponing the interrupt, till the response to the "command in flight" is received.
- Removed the mutext lock "se_if_lock".
- furnction prototype for "se_save_imem_state" and "se_restore_imem_state", is changed to pass "imem" by reference.
- Added a new structure "struct se_fw_load_info", dedicated to contain FW loading relevant info. It is a member of struct info_list.
- split "imem_mgmt_file_in_rfs" into two "prim_fw_nm_in_rfs" and "seco_fw_nm_in_rfs", to be part of "struct se_fw_load_info".
- moved the function "se_load_firmware" prior to func "if_mbox_free_channel".
- function "se_load_firmware" is updated to use "request_firmware", instead of "request_firmware_no_wait".
- function "se_load_firmware" is updated to load "primary" fw image, if the imem_state is not BAD. Then load the "secondary FW" image.
- Added a new mutex_lock in the function "se_load_firmware", for ensuring FW loading done once, when there are multiple application are in play.
- instead of "wait_queue_head_t wq", used "sruct completion".
- add devm_add_action with action as se_if_probe_cleanup.
Reference:
- Link to v7: https://lore.kernel.org/r/20240904-imx-se-if-v7-0-5afd2ab74264@nxp.com
Changes in v7:
5/5
- struct se_clbk_handle, is added with a member struct se_if_device_ctx *dev_ctx.
- func call to ele_miscdev_msg_rcv() & ele_miscdev_msg_send(), are removed.
- func se_ioctl_cmd_snd_rcv_rsp_handler(), is modified to remove the func call to ele_miscdev_msg_rcv() & ele_miscdev_msg_send()
- func se_ioctl_cmd_snd_rcv_rsp_handler is callig func ele_msg_send_rcv(), instead.
- Mutext "se_cmd_if_lock", handling is removed from this patch.
- func ele_miscdev_msg_send() is replaced with func ele_msg_send(), in fops_write.
- func ele_miscdev_msg_rcv() is replaced with func ele_msg_rcv(), in fops_read.
- fops_open is modified to create the new dev_ctx instance (using func init_device_context()), which is not registered as miscdev.
- Only one dev_ctx is registered as miscdev and its reference is stored in the struct se_if_priv, as priv_dev_ctx.
- Separate func cleanup_se_shared_mem() & func init_se_shared_mem(), for shared memory handling part of struct dev_ctx.
- Input param for func(s) ele_msg_rcv(), ele_msg_send() & ele_msg_send_rcv(), is replaced from struct se_if_priv to struct se_if_device_ctx.
4/5
- A new structure is defined name struct "se_clbk_handle", to contain members processed in mailbox call-back function.
- "struct se_if_priv" is modified to contain the two structures of "se_clbk_handle" - waiting_rsp_clbk_hdl & cmd_receiver_clbk_hdl.
- func ele_msg_rcv() is modified to take a new additional input reference param "struct se_clbk_handle *se_clbk_hdl".
- func ele_msg_send() is modified to take a new additional input tx_msg_sz.
- func ele_msg_send_rcv(), is modified to take 2 more inputs - tx_msg_sz & exp_rx_msg_sz.
- func se_val_rsp_hdr_n_status(), is modified to take input of rx_msg buffer, instead of header value, as input param.
- each caller of the func ele_msg_send_rcv(), is sending these two additional input params.
- func se_if_callback(), is modified to work on two structures of "se_clbk_handle" - waiting_rsp_clbk_hdl & cmd_receiver_clbk_hdl.
- Variable "max_dev_ctx", is removed from info & priv struture, as well its usage.
- New member variable "se_img_file_to_load", is added to structure "priv".
- Other member variables - rx_msg(ptr), rx_msg_sz, completion done & list of dev_ctxs, is removed from priv struture, along with their usage.
- func se_resume(), updated to wakeup the two "wq", part of "struct se_clbk_handle": priv->waiting_rsp_clbk_hdl & priv->cmd_receiver_clbk_hdl.
3/5
- Node name is changed from senclave-firmware@0 to "secure-enclave"
2/5
- Node name is changed to "secure-enclave".
Reference:
- Link to v6: https://lore.kernel.org/r/20240722-imx-se-if-v6-0-ee26a87b824a@nxp.com
Changes in v6:
5/5
- replaced scope_gaurd with gaurd.
4/5
- replaced scope_gaurd with gaurd.
- remove reading the regs property from dtb.
- Added NULL check for priv data fetched from device, as a sanity check, for ele_base_msg apis)
3/5
- replace firmware with senclave-firmware.
2/5
- replace firmware with senclave-firmware.
- drop description for mbox
- Replaced "items:" with maxItems:1 for "memory-region"
- Replaced "items:" with maxItems:1 for "sram"
- remove regs property.
- remove "$nodename"
Reference:
- Link to v5: https://lore.kernel.org/r/20240712-imx-se-if-v5-0-66a79903a872@nxp.com
Changes in v5:
2/5
- updated the description of mboxes
- updated the description & items for mbox-names.
- updated the description of memory-region
- move "additional properties: false" after allOf block.
- removed other example except one.
4/5
- Corrected the indentation in Kconfig.
- info members:mbox_tx_name & mbox_rx_name, are replaced with macros.
5/5
- Replaced "for secure enclaves", with "for secure enclaves"
- Replaced "user space" with "userspace".
- End the line "[include]<linux/firmware/imx/ele_mu_ioctl.h>" with a period.
Reference:
- Link to v4: https://lore.kernel.org/r/20240705-imx-se-if-v4-0-52d000e18a1d@nxp.com
Changes in v4:
1/5
a. Removed - from EdgeLock Enclave.
b. Removed , after "Each of the above feature,"
c. replace "can exists" with "can exist".
d.
-messaging units(MU) per SE. Each co-existing 'se' can have one or multiple exclusive
-MU(s), dedicated to itself. None of the MU is shared between two SEs.
+messaging units(MU) per SE. Each co-existing SE can have one or multiple exclusive
+MUs, dedicated to itself. None of the MU is shared between two SEs.
Communication of the MU is realized using the Linux mailbox driver.
e.
-All those SE interfaces 'se-if' that is/are dedicated to a particular SE, will be
-enumerated and provisioned under the very single 'SE' node.
+Although MU(s) is/are not shared between SE(s). But for SoC like i.MX95 which has
+multiple SE(s) like HSM, V2X-HSM, V2X-SHE; all the SE(s) and their interfaces 'se-if'
+that is/are dedicated to a particular SE will be enumerated and provisioned using the
+single compatible node("fsl,imx95-se").
f. Removed ",". Replaced for "Each 'se-if'," with "Each se-if'.
g. removed ","
- This layer is responsible for ensuring the communication protocol, that is defined
+ This layer is responsible for ensuring the communication protocol that is defined
h. removed "-"
- - FW can handle one command-message at a time.
+ - FW can handle one command message at a time.
i.
- Using these multiple device contexts, that are getting multiplexed over a single MU,
- user-space application(s) can call fops like write/read to send the command-message,
- and read back the command-response-message to/from Firmware.
- fops like read & write uses the above defined service layer API(s) to communicate with
+ Using these multiple device contexts that are getting multiplexed over a single MU,
+ userspace application(s) can call fops like write/read to send the command message,
+ and read back the command response message to/from Firmware.
+ fops like read & write use the above defined service layer API(s) to communicate with
Firmware.
j. Uppercase for word "Linux".
2/5
a. Rephrased the description to remove list of phandles.
b. Moved required before allOf:
+required:
+ - compatible
+ - reg
+ - mboxes
+ - mbox-names
+
+additionalProperties: false
+
allOf:
c. replaced not: required: with properties: <property-name>: false.
# memory-region
- not:
- required:
- - memory-region
+ properties:
+ memory-region: false
# sram
- else:
- not:
- required:
- - sram
d. Reduced examples. keeping example of i.MX95.
e. node-name is changed to "firmware@<hex>"
3/5
- node name changed to "firmware@<hex>".
4/5
- used sizeof(*s_info)
- return early, rather than doing goto exit, in ele_get_info().
- Use upper_32_bits() and lower_32_bits()
- use rx_msg here instead of priv->rx_msg
- Moved the status check to validate_rsp_hdr. Rename the function to "se_val_rsp_hdr_n_status"
- typecasting removed header = (struct se_msg_hdr *) msg;
- Converted the API name with prefix imx_ele_* or imx_se_*, to ele_* and se_*, respectively.
- Removed the functions definition & declaration for: free_phybuf_mem_pool() & get_phybuf_mem_pool()
- removed the mbox_free_channel() calls from clean-up.
- Flag "priv->flags" is removed.
- Converted the int se_if_probe_cleanup() to void se_if_probe_cleanup().
- Replaced NULL initialization of structure members: priv->cmd_receiver_dev & priv->waiting_rsp_dev , with comments.
- Removed the function's declaration get_phy_buf_mem_pool1
5/5
Changes to Documentation/ABI/testing/se-cdev.
a. Removed "-" from "secure-enclave" and "file-descriptor".
b. Removed "-" from "shared-library"
c. Replaced "get" with "getting".
d. Added description for the new IOCTL "send command and receive command response"
e. Replaced "wakeup_intruptible" with "wait_event_interruptible"
f. Removed ";"
g. Removd "," from "mailbox_lock,"
h. Replaced "free" with "frees"
i. In mailbox callback function, checking the buffer size before
copying.
Reference:
- Link to v3: https://lore.kernel.org/r/20240617-imx-se-if-v3-0-a7d28dea5c4a@nxp.com
Changes in v3:
5/5:
- Initialize tx_msg with NULL.
- memdup_user() returns an error pointer, not NULL. correct it by adding check for err_ptr.
- new IOCTL is added to send & recieve the message.
- replaced the while loop till list is empty, with list_for_each_entry.
- replaced __list_del_entry, with list_del.
- Removed the dev_err message from copy to user.
- Removed the casting of void *.
- corrected the typcasting in copy to user.
- removed un-necessary goto statement.
- Removed dead code for clean-up of memory.
- Removed un-mapping of secured memory
- Passing se_if_priv structure to init_device_context.
- Updated the below check to replace io.length with round_up(io.length).
if (shared_mem->size < shared_mem->pos|| io.length >= shared_mem->size - shared_mem->pos)
- Created a function to cleanup the list of shared memory buffers.
- Used list_for_each_entry_safe(). created a separate functions: se_dev_ctx_cpy_out_data() & se_dev_ctx_shared_mem_cleanup()
4/5
- Changed the compatible string to replace "-ele", to "-se".
- Declaration of imx_se_node_info, is done as const in the whole file
- Remove the unused macros from ele_base_msg.h
- Remove the function declaration get_phy_buf_mem_pool1, from the header file.
- Replace the use of dmam_alloc_coherent to dma_alloc_coherent
- Check for function pointer, before calling the fucntion pointer in imx_fetch_se_soc_info
- Removed the unused flag for SE_MU_IO_FLAGS_USE_SEC_MEM.
- Removed the unused macros WORD_SZ
- instead of struct device *dev, struct se_if_priv *priv, is used as argument to the funtions:se_save_imem_state, se_restore_imem_state, imx_fetch_se_soc_info
- Removed ret from validate_rsp_hdr.
- changed the prefix of the funtion: plat_add_msg_crc and plat_fill_cmd_msg_hdr.
- indentation correction for info structures.
- remove the check for priv not null from se_if_probe_cleanup
- Removed the casting of void *.
- se_load_firmware function is corrected for not freeing the buffer when allocation fails.
- Checking if get_imx_se_node_info() can return NULL, in se_if_probe()
- imem.size has type u32. return value from se_save_imem_state() will be assigned to imem.size in case of success only.
- removed the flag un-setting in case of failure. priv->flags &= (~RESERVED_DMA_POOL);
- removed the function call for devm_of_platform_populate(dev);
- Checking for not-NULL, before calling the funtion pointer se_fetch_soc_info.
- Removed the checking for reserved memory flag, before freeing up the reserved memory, in se_probe_if_cleanup.
3/5
- Changed the compatible string to replace "-ele", to "-se".
2/5
- to fix the warning error, replaced the "-ele" & "-v2x" in compatible string, to "-se".
- Added an example for ele@0 for compatible string "fsl,imx95-se"
Reference
- Link to v2: https://lore.kernel.org/r/20240523-imx-se-if-v2-0-5a6fd189a539@nxp.com
Changes in v2:
4/4
- Split this patch into two: 1. base driver & 2. Miscdev
- Initialize the return variable "err" as 0, before calling 'return err', in the file ele_common.c
- Fix the usage of un-iniitialized pointer variable, by initializing them with NULL, in ele_base_msg.c.
- Fix initializing the ret variable, to return the correct error code in case of issue.
- replaced dmam_alloc_coherent with dma_alloc_coherent.
- Replace the use of ELE_GET_INFO_READ_SZ, with sizeof(soc_info).
- Replaced -1 with -EPERM
- Removed the safety check on func-input param, in ele_get_info().
- fix the assigning data[1] with lower 32 address, rather than zero, for ele_fw_authenticate API.
- Correctly initializing the function's return error code, for file ele_base_msg.c.
- replaced 'return' with 'goto'.
- Use length in bytes.
- Corrected the structure se_msg_hdr.
- Moved setting of rx_msg to priv, into the function imx_ele_msg_send_rcv
- Will add lockdep_assert_held, to receive path, in v2.
- corrected the spacing at "ret = validate_rsp_hdr"
- FIELD_GET() used for RES_STATUS
- Re-write the structure soc_info, matching the information provided in response to this api.
- The "|" goes to the end of the previous line.
- Moved the locking and unlocking of the command lock to the caller of the function.
- removed the safety check for device private data.
- Structure memory reference, used to read message header.
- In the interrupt call back function, remove assigning waiting_rsp_dev to NULL, in case of response message rcv from FW.
- do while removed.
- replaced BIT(1) for RESERVED_DMA_POOL, to BIT(0)
- The backslash is removed while assigning the file name with absolute path to structure variable.fw_name_in_rfs =.
- Update the 'if' condition by removing "idx < 0".
- mbox_request_channel_byname() uses a "char" for the name not a u8. Corrected.
- devm managed resources, are not cleaned now, in function se_probe_if_cleanup
- Used dev_err_probe().
- Used %pe to print error string.
- remove "__maybe_unused" for "struct platform_device *enum_plat_dev __maybe_unused;"
- used FIELD_GET(), for RES_STATUS. Removed the use of MSG_TAG, MSG_COMMAND, MSG_SIZE, MSG_VER.
- Depricated the used of member of struct se_if_priv, bool no_dev_ctx_used;
- Moved the text explaing the synchronization logic via mutexes, from patch 1/4 to se_ctrl.h.
- removed the type casting of info_list = (struct imx_se_node_info_list *) device_get_match_data(dev->parent);
- Used static variable priv->soc_rev in the se_ctrl.c, replaced the following condition: if (info_list->soc_rev) to if (priv->soc_rev) for checking if this flow is already executed or not.
- imx_fetch_soc_info will return failure if the get_info function fails.
- Removed devm_free from imx_fetch_soc_info too.
3/3
- Made changes to move all the properties to parent node, without any child node.
2/4
- Use Hex pattern string.
- Move the properties to parent node, with no child node.
- Add i.MX95-ele to compatible nodes to fix the warning "/example-2/v2x: failed to match any schema with compatible: ['fsl,imx95-v2x']"
1/1
- Corrected the spelling from creats to creates.
- drop the braces around the plural 's' for interfaces
- written se in upper case SE.
- Replace "multiple message(s)" with messages.
- Removed too much details about locks.
Testing
- make CHECK_DTBS=y freescale/imx8ulp-evk.dtb;
- make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 dt_binding_check DT_SCHEMA_FILES=fsl,imx-se.yaml
- make C=1 CHECK=scripts/coccicheck drivers/firmware/imx/*.* W=1 > r.txt
- ./scripts/checkpatch.pl --git <>..HEAD
- Tested the Image and .dtb, on the i.MX8ULP.
Reference
- Link to v1: https://lore.kernel.org/r/20240510-imx-se-if-v1-0-27c5a674916d@nxp.com
---
Pankaj Gupta (7):
Documentation/firmware: add imx/se to other_interfaces
dt-bindings: arm: fsl: add imx-se-fw binding doc
firmware: imx: add driver for NXP EdgeLock Enclave
firmware: imx: device context dedicated to priv
firmware: drivers: imx: adds miscdev
arm64: dts: imx8ulp: add secure enclave node
arm64: dts: imx8ulp-evk: add reserved memory property
Documentation/ABI/testing/se-cdev | 44 +
.../devicetree/bindings/firmware/fsl,imx-se.yaml | 91 ++
.../driver-api/firmware/other_interfaces.rst | 133 ++
arch/arm64/boot/dts/freescale/imx8ulp-evk.dts | 12 +-
arch/arm64/boot/dts/freescale/imx8ulp.dtsi | 11 +-
drivers/firmware/imx/Kconfig | 13 +
drivers/firmware/imx/Makefile | 2 +
drivers/firmware/imx/ele_base_msg.c | 307 ++++
drivers/firmware/imx/ele_base_msg.h | 117 ++
drivers/firmware/imx/ele_common.c | 540 +++++++
drivers/firmware/imx/ele_common.h | 53 +
drivers/firmware/imx/ele_fw_api.c | 57 +
drivers/firmware/imx/ele_fw_api.h | 18 +
drivers/firmware/imx/se_ctrl.c | 1676 ++++++++++++++++++++
drivers/firmware/imx/se_ctrl.h | 190 +++
include/linux/firmware/imx/se_api.h | 14 +
include/uapi/linux/se_ioctl.h | 97 ++
17 files changed, 3372 insertions(+), 3 deletions(-)
---
base-commit: 78be1e893a8196d509f48064dd7370274f8851b8
change-id: 20240507-imx-se-if-a40055093dc6
Best regards,
--
Pankaj Gupta <pankaj.gupta@nxp.com>
^ permalink raw reply
* RE: [External Mail] Re: [PATCH v4 2/7] net: wwan: t9xx: Add control plane transaction layer
From: Wu. JackBB (GSM) @ 2026-07-17 10:53 UTC (permalink / raw)
To: Simon Horman
Cc: loic.poulain@oss.qualcomm.com, ryazanov.s.a@gmail.com,
johannes@sipsolutions.net, andrew+netdev@lunn.ch,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
pabeni@redhat.com, wen-zhi.huang@mediatek.com,
shi-wei.yeh@mediatek.com, Minano.tseng@mediatek.com,
matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com,
corbet@lwn.net, skhan@linuxfoundation.org,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org, linux-doc@vger.kernel.org
In-Reply-To: <47002e869f5b4f0985316669d2769c19@compal.com>
Hi Simon,
After further internal review, we agree this needs to be fixed.
On Jack Wu wrote:
>> [Severity: High]
>> Does this code apply a double byte-swap on big-endian architectures?
>T9XX currently only targets x86 (little-endian) platforms where
>cpu_to_le32() is a no-op. The Kconfig enforces "depends on PCI &&
In v5 we will remove LE32_TO_U32(cpu_to_le32(...)) and return
hw_bits directly in both mtk_pci_ext_d2h_evt_hw_bits() and
mtk_pci_ext_h2d_evt_hw_bits(). The custom LE32_TO_U32 macro
will also be removed.
Thanks.
Jack Wu
^ permalink raw reply
* RE: [External Mail] Re: [PATCH v4 1/7] net: wwan: t9xx: Add PCIe core
From: Wu. JackBB (GSM) @ 2026-07-17 10:51 UTC (permalink / raw)
To: Simon Horman
Cc: loic.poulain@oss.qualcomm.com, ryazanov.s.a@gmail.com,
johannes@sipsolutions.net, andrew+netdev@lunn.ch,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
pabeni@redhat.com, wen-zhi.huang@mediatek.com,
shi-wei.yeh@mediatek.com, Minano.tseng@mediatek.com,
matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com,
corbet@lwn.net, skhan@linuxfoundation.org,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org, linux-doc@vger.kernel.org
In-Reply-To: <93775de437ca4e25bb59b58a3d083dd0@compal.com>
Hi Simon,
After further internal review, we agree this needs to be fixed.
On Jack Wu wrote:
>> [Severity: Medium]
>> Will this break channel index calculation on big-endian systems?
> This driver targets MediaTek T9xx PCIe WWAN modems on x86/ARM64-LE
> platforms only. On little-endian, cpu_to_le32() is a no-op and
> LE32_TO_U32() is a simple cast — no byte-swap occurs. The Kconfig
> enforces depends on PCI && ACPI, effectively restricting to LE
> platforms where this hardware exists.
In v5 we will:
- Remove LE32_TO_U32(cpu_to_le32(...)) and return hw_bits
directly in both mtk_pci_ext_d2h_evt_hw_bits() and
mtk_pci_ext_h2d_evt_hw_bits()
- Remove the custom LE32_TO_U32 macro
Thanks.
寄件者: Wu. JackBB (GSM)
寄件日期: 2026年7月17日 下午 02:25
收件者: Simon Horman
副本: loic.poulain@oss.qualcomm.com; ryazanov.s.a@gmail.com; johannes@sipsolutions.net; andrew+netdev@lunn.ch; davem@davemloft.net; edumazet@google.com; kuba@kernel.org; pabeni@redhat.com; wen-zhi.huang@mediatek.com; shi-wei.yeh@mediatek.com; Minano.tseng@mediatek.com; matthias.bgg@gmail.com; angelogioacchino.delregno@collabora.com; corbet@lwn.net; skhan@linuxfoundation.org; linux-kernel@vger.kernel.org; netdev@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-doc@vger.kernel.org
主旨: RE: [External Mail] Re: [PATCH v4 1/7] net: wwan: t9xx: Add PCIe core
Hi Simon,
> > +static irqreturn_t mtk_pci_irq_msix(int irq, void *data)
> > +{
> > + if (unlikely(!irq_state) ||
> > + unlikely(!((irq_state & GENMASK(priv->irq_cnt - 1, 0)) &
> > + irq_desc->msix_bits)))
> > + return IRQ_NONE;
>
> [Severity: High]
> Does this logic silently drop hardware interrupts when fewer than 32
> vectors are allocated?
No interrupts are dropped. This is the MSI-X merged mode design.
When fewer vectors are allocated, mtk_pci_set_msix_merged() configures
the hardware to merge all interrupt sources into the available vectors.
For example, with irq_cnt=1, the hardware maps ALL sources (including
MHCCIF at hardware bit 28) to vector 0. The MSI-X status register
shows BIT(0), not BIT(28). So irq_desc[0].msix_bits = BIT(0), and
GENMASK(0,0) & BIT(0) & BIT(0) = 1 — the check passes correctly.
The GENMASK filters to allocated vector bits only, then checks if the
current handler's vector is set. This works for both full allocation
(32 vectors, 1:1 mapping) and merged mode (fewer vectors, N:1 mapping).
> > + ret = pci_request_irq(pdev, i, mtk_pci_irq_msix, NULL,
> > + &irq_desc[i], irq_desc[i].name);
>
> [Severity: Medium]
> Could this trigger a -Wformat-security compiler warning?
We will change to pci_request_irq(..., "%s",
irq_desc[i].name) to avoid format string issues.
> > + hw_bits = mtk_pci_ext_h2d_evt_hw_bits(ch);
> > + mtk_pci_write32(mdev, rc_base + MHCCIF_RC2EP_SW_TCHNUM,
> > + ffs(hw_bits) - 1);
>
> [Severity: Medium]
> Will this break channel index calculation on big-endian systems?
This driver targets MediaTek T9xx PCIe WWAN modems on x86/ARM64-LE
platforms only. On little-endian, cpu_to_le32() is a no-op and
LE32_TO_U32() is a simple cast — no byte-swap occurs. The Kconfig
enforces depends on PCI && ACPI, effectively restricting to LE
platforms where this hardware exists.
> [Severity: Medium]
> What happens if the driver attempts to send an unmapped channel?
All callers of mtk_pci_send_ext_evt() pass known-valid channel values
that are mapped in SET_HW_BITS. The is_power_of_2(ch) check at the
top of the function rejects invalid input. The FSM single-thread
design ensures all ext_evt sends go through a controlled path, so
an unmapped channel cannot reach ffs().
> > + priv->ext_reg_base = priv->bar23_addr - ATR_PCIE_REG_TRSL_ADDR;
>
> [Severity: Medium]
> Is it safe to store an out-of-bounds pointer?
This is a standard pattern in PCIe MMIO drivers using Address
Translation Registers (ATR). The modem's internal register space
starts at address 0x0 in its own view, but the host accesses it
through BAR2/3 mapped at offset ATR_PCIE_REG_TRSL_ADDR (0x10000000).
Pre-computing ext_reg_base = bar23_addr - 0x10000000 allows using
modem register addresses directly: ioread32(ext_reg_base + reg_addr).
ext_reg_base is typed as void __iomem *, an opaque cookie for MMIO
access — never dereferenced as a regular pointer. The ioread32/
iowrite32 functions perform platform-specific MMIO access on it.
UBSAN does not flag __iomem arithmetic. This same pattern is used
by other kernel PCIe drivers (e.g., mtk_t7xx).
> > +static void mtk_pci_remove(struct pci_dev *pdev)
> > +{
> > + if (mtk_pci_pldr(mdev)) { ... }
> > + pci_clear_master(pdev);
> > + mtk_pci_free_irq(mdev);
> > + mtk_mhccif_exit(mdev);
>
> [Severity: High]
> Does this sequence result in the device being powered off before
> the OS interrupt handlers and workqueues are synchronized?
The device is not left powered off. mtk_pci_pldr() calls ACPI
PXP._OFF then PXP._ON — the _ON call restores power and brings
the PCIe link back up. After _ON returns, the device is powered
on and BARs are accessible.
The PLDR sequence is: (1) _OFF: power-cycle modem firmware,
(2) msleep for power drain, (3) _ON: restore power and link.
After step 3, the device is live with a fresh firmware state.
Additionally, before PLDR, mtk_pci_mask_irq() disables the MHCCIF
interrupt at hardware level, preventing new interrupts. Between
pci_clear_master and mtk_pci_free_irq, mtk_pci_dev_exit() tears
down FSM and trans_ctrl, ensuring no workqueues remain scheduled.
> > + /* Request a slot reset. */
> > + return PCI_ERS_RESULT_CAN_RECOVER;
>
> [Severity: Medium]
> Will returning PCI_ERS_RESULT_CAN_RECOVER actually abort the slot
> reset?
We will change to PCI_ERS_RESULT_DISCONNECT and update
the comment, since the driver does not currently support AER
recovery.
Thanks.
Jack Wu
^ permalink raw reply
* [RFC PATCH v1 8/8] misc/arm-cla: Add userspace interface
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
Expose CLA devices through a character device so userspace can enumerate
the available hardware and map accelerator register frames.
Define version 1 of the CLA UAPI with a GET_PARAM ioctl. Report device
topology, CPU affinity, domain membership, mmap offsets, architecture
version and attached accelerator masks, together with the IIDR, DEVARCH
and REVIDR of each accelerator.
CLA registers can only be read from the CPU local to the device, while
enumeration may occur on any CPU. Validate the supported CLA
architecture version during device setup and cache the CLA and
accelerator identification registers for later ioctl queries.
Support shared read-write mmap of one or more CLA register pages. Create
a context for every domain covered by the mapping and resolve faults
only while that context owns the domain. Queue unassigned contexts with
the domain scheduler, drop mmap_lock while waiting for assignment and
retry the fault after the context is woken.
Co-developed-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
drivers/misc/arm-cla/Makefile | 3 +-
drivers/misc/arm-cla/arm-cla-regs.h | 2 +
drivers/misc/arm-cla/arm-cla.h | 13 ++
drivers/misc/arm-cla/cla-init.c | 26 +++
drivers/misc/arm-cla/cla-user.c | 351 ++++++++++++++++++++++++++++
include/uapi/linux/arm-cla.h | 207 ++++++++++++++++
6 files changed, 601 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/arm-cla/cla-user.c
create mode 100644 include/uapi/linux/arm-cla.h
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index a92c3594e5c9..f848c608e8f1 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -7,6 +7,7 @@ arm-cla-y := \
cla-ops.o \
cla-regs.o \
cla-sched.o \
- cla-topology.o
+ cla-topology.o \
+ cla-user.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla-regs.h b/drivers/misc/arm-cla/arm-cla-regs.h
index fcd187ecb777..338b9e78b465 100644
--- a/drivers/misc/arm-cla/arm-cla-regs.h
+++ b/drivers/misc/arm-cla/arm-cla-regs.h
@@ -63,6 +63,8 @@
CLA_STATUS_EXCEPT | \
CLA_STATUS_SRMODE)
+#define CLA_AAIDR_1_0 (1ULL << 8)
+
/* Standard accelerator registers */
#define CLA_REG_IIDR 0x0000
#define CLA_REG_DEVARCH 0x0001
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 041d4a8d7e91..3dd783d0b1cc 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -45,6 +45,12 @@ struct cla_call_on_cpu {
struct kthread_work switch_ctx;
};
+struct cla_accel_desc {
+ u64 iidr;
+ u64 devarch;
+ u64 revidr;
+};
+
/**
* struct cla_dev - CLA device
*
@@ -57,6 +63,8 @@ struct cla_call_on_cpu {
* @pg_offset: Mmap offset of this device.
* @iassizes: Combined regstate of all accels.
* @domain: The domain this CLA belongs to.
+ * @aidr: CLA Architecture ID.
+ * @accel_descs: Accelerator ID registers.
*
* Mutable, only accessed under @lock:
* @lock: Protects the following members.
@@ -74,6 +82,8 @@ struct cla_dev {
unsigned long pg_offset;
unsigned long iassizes;
struct cla_domain *domain;
+ u64 aidr;
+ struct cla_accel_desc accel_descs[CLA_NUM_ACC];
struct mutex lock;
bool broken;
@@ -280,4 +290,7 @@ int cla_mtc_clear(struct cla_dev *dev);
int cla_mtc_install(struct cla_dev *dev, pgd_t *pgd, unsigned long asid);
int cla_mtc_uninstall(struct cla_dev *dev);
+int cla_user_init(void);
+void cla_user_exit(void);
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 5335c55568ad..090efc6cc9b4 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -80,6 +80,7 @@ static int cla_reset_pmu(struct cla_dev *dev, unsigned int accid)
*/
static int cla_dev_setup_accel(struct cla_dev *dev, unsigned int accid)
{
+ struct cla_accel_desc *desc = &dev->accel_descs[accid];
u64 status;
u64 iassize;
u64 acap;
@@ -122,6 +123,20 @@ static int cla_dev_setup_accel(struct cla_dev *dev, unsigned int accid)
return 1;
}
+ /*
+ * Cache some standard accelerator registers that user space may query
+ * via ioctl from a remote CPU.
+ */
+ ret = cla_op_regread(dev, accid, CLA_REG_IIDR, 1, &desc->iidr);
+ if (ret)
+ return ret;
+ ret = cla_op_regread(dev, accid, CLA_REG_DEVARCH, 1, &desc->devarch);
+ if (ret)
+ return ret;
+ ret = cla_op_regread(dev, accid, CLA_REG_REVIDR, 1, &desc->revidr);
+ if (ret)
+ return ret;
+
ret = cla_op_regread(dev, accid, CLA_REG_IASSIZE, 1, &iassize);
if (ret)
return ret;
@@ -234,6 +249,10 @@ static int cla_dev_setup(unsigned int cpu)
if (WARN_ON(smp_processor_id() != cpu || dev->cpu != cpu))
return -EINVAL;
+ dev->aidr = cla_reg_read(dev, CLA_REG_CLAAIDR);
+ if (dev->aidr != CLA_AAIDR_1_0)
+ return -EPROTONOSUPPORT;
+
/* Clear DATA and LRESP_DATANZ */
for (i = 0; i < CLA_NUM_DATA_REGS; i++)
cla_reg_write(dev, CLA_REG_DATA(i), 0);
@@ -453,8 +472,14 @@ static int __init cla_module_init(void)
}
cla_cpuhp_state = ret;
+ ret = cla_user_init();
+ if (ret)
+ goto err_cpuhp_remove;
+
return 0;
+err_cpuhp_remove:
+ cpuhp_remove_state(cla_cpuhp_state);
err_driver_unregister:
platform_driver_unregister(&cla_driver);
err_domains_free:
@@ -464,6 +489,7 @@ static int __init cla_module_init(void)
static void __exit cla_module_exit(void)
{
+ cla_user_exit();
cpuhp_remove_state(cla_cpuhp_state);
platform_driver_unregister(&cla_driver);
cla_domains_free();
diff --git a/drivers/misc/arm-cla/cla-user.c b/drivers/misc/arm-cla/cla-user.c
new file mode 100644
index 000000000000..415328a653b7
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-user.c
@@ -0,0 +1,351 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm CLA driver - userspace interface
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/cdev.h>
+#include <linux/cpu.h>
+#include <linux/device.h>
+#include <linux/fs.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include "arm-cla.h"
+#include <uapi/linux/arm-cla.h>
+
+static struct class *cla_class;
+static dev_t cla_devt;
+
+#define dev_nospec(idx) array_index_nospec(idx, cla_nr_devs)
+#define accel_nospec(idx) array_index_nospec(idx, CLA_NUM_ACC)
+
+#define cla_for_each_mapped_domain(_vma, _domain) \
+ for (unsigned long __pg = (_vma)->vm_pgoff; \
+ __pg < (_vma)->vm_pgoff + vma_pages(_vma) && \
+ ((_domain) = cla_lut_pg[dev_nospec(__pg)]->domain); \
+ __pg = (_domain)->pg_offset + (_domain)->nr_devs)
+
+static long cla_ioctl_validate_param(struct arm_cla_param *param)
+{
+ int accel_id;
+ int dev_id;
+
+ switch (param->param) {
+ case ARM_CLA_PARAM_UABI_VERSION:
+ case ARM_CLA_PARAM_DEV_NR:
+ if (param->index != 0)
+ return -EINVAL;
+ break;
+ case ARM_CLA_PARAM_DEV_CPU_ID:
+ case ARM_CLA_PARAM_DEV_DOMAIN_ID:
+ case ARM_CLA_PARAM_DEV_PGOFF:
+ case ARM_CLA_PARAM_DEV_AIDR:
+ case ARM_CLA_PARAM_DEV_ACCELS:
+ if (param->index >= cla_nr_devs)
+ return -EINVAL;
+ break;
+ case ARM_CLA_PARAM_ACCEL_IIDR:
+ case ARM_CLA_PARAM_ACCEL_DEVARCH:
+ case ARM_CLA_PARAM_ACCEL_REVIDR:
+ dev_id = ARM_CLA_PARAM_INDEX_DEV(param->index);
+ accel_id = ARM_CLA_PARAM_INDEX_ACCEL(param->index);
+ if (dev_id >= cla_nr_devs || accel_id >= CLA_NUM_ACC)
+ return -EINVAL;
+ dev_id = dev_nospec(dev_id);
+ if ((cla_lut_pg[dev_id]->accelerators & BIT(accel_id)) == 0)
+ return -ENODEV;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static long cla_ioctl_get_param(unsigned long arg)
+{
+ struct arm_cla_param __user *uparam = (void __user *)arg;
+ struct arm_cla_param param;
+ int accel_id;
+ int dev_id;
+ int ret;
+
+ if (copy_from_user(¶m, uparam, sizeof(param)))
+ return -EFAULT;
+
+ ret = cla_ioctl_validate_param(¶m);
+ if (ret)
+ return ret;
+
+ dev_id = dev_nospec(ARM_CLA_PARAM_INDEX_DEV(param.index));
+ accel_id = accel_nospec(ARM_CLA_PARAM_INDEX_ACCEL(param.index));
+
+ switch (param.param) {
+ case ARM_CLA_PARAM_UABI_VERSION:
+ param.value = ARM_CLA_UABI_VERSION;
+ break;
+ case ARM_CLA_PARAM_DEV_NR:
+ param.value = cla_nr_devs;
+ break;
+ case ARM_CLA_PARAM_DEV_CPU_ID:
+ param.value = cla_lut_pg[dev_id]->cpu;
+ break;
+ case ARM_CLA_PARAM_DEV_DOMAIN_ID:
+ param.value = cla_lut_pg[dev_id]->domain->id;
+ break;
+ case ARM_CLA_PARAM_DEV_PGOFF:
+ param.value = cla_lut_pg[dev_id]->pg_offset;
+ break;
+ case ARM_CLA_PARAM_DEV_AIDR:
+ param.value = cla_lut_pg[dev_id]->aidr;
+ break;
+ case ARM_CLA_PARAM_DEV_ACCELS:
+ param.value = cla_lut_pg[dev_id]->accelerators;
+ break;
+ case ARM_CLA_PARAM_ACCEL_IIDR:
+ param.value = cla_lut_pg[dev_id]->accel_descs[accel_id].iidr;
+ break;
+ case ARM_CLA_PARAM_ACCEL_DEVARCH:
+ param.value = cla_lut_pg[dev_id]->accel_descs[accel_id].devarch;
+ break;
+ case ARM_CLA_PARAM_ACCEL_REVIDR:
+ param.value = cla_lut_pg[dev_id]->accel_descs[accel_id].revidr;
+ break;
+ }
+
+ if (copy_to_user(uparam, ¶m, sizeof(param)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static void cla_vma_open(struct vm_area_struct *vma)
+{
+ /*
+ * A vma previously created with cla_file_mmap() has been duplicated
+ * within the same mm (most likely due to mremap). While this could also
+ * be called for duplication into a new mm (via fork), we set
+ * VM_DONTCOPY on the original mmap, so this will never happen. So the
+ * contexts covered by this new vma already exist.
+ */
+ struct cla_domain *domain;
+
+ cla_for_each_mapped_domain(vma, domain)
+ cla_ctx_map(domain, vma->vm_mm, vma->vm_file);
+}
+
+static void cla_vma_close(struct vm_area_struct *vma)
+{
+ struct cla_domain *domain;
+
+ /* On munmap() or exit_mmap(), kill the context. */
+ cla_for_each_mapped_domain(vma, domain)
+ cla_ctx_unmap(domain, vma->vm_mm, vma->vm_file);
+}
+
+static vm_fault_t cla_vma_fault(struct vm_fault *vmf)
+{
+ struct vm_area_struct *vma = vmf->vma;
+ struct cla_domain *domain;
+ struct cla_dev *dev;
+ struct cla_ctx *ctx;
+ unsigned long pg;
+
+ /*
+ * Reassignment needs mmap_lock, so we cannot wait while holding it.
+ * Luckily, this flag is only missing in some exotic cases which do not
+ * apply for this VM_IO | VM_PFNMAP vma. i.e. GUP without
+ * FOLL_UNLOCKABLE or FOLL_NOWAIT or fixup_user_fault(unlocked=NULL),
+ * used by futex fault-in path.
+ */
+ if (WARN_ON(!(vmf->flags & FAULT_FLAG_ALLOW_RETRY)))
+ return VM_FAULT_SIGBUS;
+
+ pg = vma->vm_pgoff + ((vmf->address - vma->vm_start) >> PAGE_SHIFT);
+ dev = cla_lut_pg[pg];
+ domain = dev->domain;
+
+ mutex_lock(&domain->lock);
+ ctx = cla_domain_lookup_ctx(domain, vma->vm_mm, vma->vm_file);
+ if (WARN_ON(!ctx) || cla_ctx_is_dying(ctx) || domain->broken) {
+ mutex_unlock(&domain->lock);
+ return domain->broken ? VM_FAULT_SIGBUS : VM_FAULT_SIGSEGV;
+ }
+
+ /* If our ctx is the assigned one, map the device into memory. */
+ if (domain->assigned_ctx == ctx) {
+ vm_fault_t ret;
+
+ ret = vmf_insert_pfn(vma, vmf->address, dev->pfn);
+ mutex_unlock(&domain->lock);
+ return ret;
+ }
+
+ /* Enqueue if not already, starting current assignee's time slice. */
+ if (list_empty(&ctx->queue_node)) {
+ bool was_empty = list_empty(&domain->queued_ctxs);
+ unsigned long delay;
+
+ list_add_tail(&ctx->queue_node, &domain->queued_ctxs);
+ if (was_empty) {
+ delay = domain->assigned_ctx ? CLA_SLICE_MS : 0;
+ cla_domain_schedule_reassignment(domain, delay);
+ }
+ }
+
+ mutex_unlock(&domain->lock);
+
+ /* If waiting is not permitted, return indicating we need to retry. */
+ if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT)
+ return VM_FAULT_RETRY;
+
+ /*
+ * Drop the fault lock prior to sleeping and return VM_FAULT_RETRY when
+ * we wake up to rerun the fault. We must get the ctx while sleeping to
+ * prevent it from being freed while we are asleep on the contained wait
+ * queue.
+ */
+ cla_ctx_get(ctx);
+ release_fault_lock(vmf);
+ wait_event_interruptible(ctx->waitq,
+ READ_ONCE(domain->assigned_ctx) == ctx ||
+ cla_ctx_is_dying(ctx) ||
+ READ_ONCE(domain->broken));
+ cla_ctx_put(ctx);
+ return VM_FAULT_RETRY;
+}
+
+static int cla_vma_may_split(struct vm_area_struct *vma, unsigned long addr)
+{
+ /*
+ * Forbid splitting cla mappings to prevent refcount leaks.
+ * cla_file_mmap()/cla_vma_open() track a context per domain mapped by
+ * the vma. If there were fewer domains mapped by the vma at
+ * cla_vma_close() then contexts would get leaked.
+ */
+ return -EINVAL;
+}
+
+static const struct vm_operations_struct cla_vma_ops = {
+ .open = cla_vma_open,
+ .close = cla_vma_close,
+ .fault = cla_vma_fault,
+ .may_split = cla_vma_may_split,
+};
+
+static int cla_file_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ unsigned long nr_pages = vma_pages(vma);
+ struct cla_domain *rollback;
+ struct cla_domain *domain;
+ struct cla_ctx *ctx;
+
+ /* Ensure the requested mapping is within range. */
+ if (!nr_pages)
+ return -EINVAL;
+ if (vma->vm_pgoff >= cla_nr_devs)
+ return -EINVAL;
+ if (nr_pages > cla_nr_devs - vma->vm_pgoff)
+ return -EINVAL;
+
+ /* Only allow shared RW mappings. Nothing else makes sense for CLA. */
+ if (!(vma->vm_flags & VM_SHARED))
+ return -EINVAL;
+ if (!(vma->vm_flags & VM_READ))
+ return -EINVAL;
+ if (!(vma->vm_flags & VM_WRITE))
+ return -EINVAL;
+ if (vma->vm_flags & VM_EXEC)
+ return -EINVAL;
+
+ /*
+ * Iterate over each domain covered by the vma and get-or-alloc its
+ * context. If any fails, we need to rollback with a put.
+ */
+ cla_for_each_mapped_domain(vma, domain) {
+ ctx = cla_ctx_map(domain, vma->vm_mm, vma->vm_file);
+ if (IS_ERR(ctx)) {
+ cla_for_each_mapped_domain(vma, rollback) {
+ if (rollback == domain)
+ return PTR_ERR(ctx);
+ cla_ctx_unmap(rollback, vma->vm_mm,
+ vma->vm_file);
+ }
+ }
+ }
+
+ vm_flags_mod(vma, VM_DONTCOPY | VM_DONTDUMP | VM_DONTEXPAND |
+ VM_IO | VM_PFNMAP, VM_MAYEXEC);
+ vma->vm_page_prot = pgprot_device(vma->vm_page_prot);
+ vma->vm_ops = &cla_vma_ops;
+
+ return 0;
+}
+
+static long cla_file_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ switch (cmd) {
+ case ARM_CLA_IOCTL_GET_PARAM:
+ return cla_ioctl_get_param(arg);
+ default:
+ return -ENOTTY;
+ }
+}
+
+static const struct file_operations cla_fops = {
+ .owner = THIS_MODULE,
+ .mmap = cla_file_mmap,
+ .unlocked_ioctl = cla_file_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = cla_file_ioctl,
+#endif
+};
+
+static char *cla_devnode(const struct device *dev, umode_t *mode)
+{
+ if (mode)
+ *mode = 0666;
+ return NULL;
+}
+
+int __init cla_user_init(void)
+{
+ int ret;
+
+ cla_class = class_create("arm-cla");
+ if (IS_ERR(cla_class))
+ return PTR_ERR(cla_class);
+ cla_class->devnode = cla_devnode;
+
+ ret = register_chrdev(0, KBUILD_MODNAME, &cla_fops);
+ if (ret < 0)
+ goto err_class_destroy;
+
+ cla_devt = MKDEV(ret, 0);
+
+ if (IS_ERR(device_create(cla_class, NULL, cla_devt, NULL, "cla"))) {
+ ret = -ENODEV;
+ goto err_unregister_chrdev;
+ }
+
+ return 0;
+
+err_unregister_chrdev:
+ unregister_chrdev(MAJOR(cla_devt), KBUILD_MODNAME);
+err_class_destroy:
+ class_destroy(cla_class);
+ return ret;
+}
+
+void __exit cla_user_exit(void)
+{
+ device_destroy(cla_class, cla_devt);
+ unregister_chrdev(MAJOR(cla_devt), KBUILD_MODNAME);
+ class_destroy(cla_class);
+}
diff --git a/include/uapi/linux/arm-cla.h b/include/uapi/linux/arm-cla.h
new file mode 100644
index 000000000000..6357fdb9df8a
--- /dev/null
+++ b/include/uapi/linux/arm-cla.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * UAPI definitions for the CLA character device.
+ */
+
+#ifndef _UAPI_LINUX_CLA_H
+#define _UAPI_LINUX_CLA_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+#define ARM_CLA_UABI_VERSION 1
+#define ARM_CLA_IOC_MAGIC 'C'
+
+/**
+ * define ARM_CLA_IOCTL_GET_PARAM - Get the value of a parameter.
+ *
+ * ioctl command whose argument is a pointer to &struct arm_cla_param.
+ * &arm_cla_param->param and &arm_cla_param->index are input parameters.
+ * &arm_cla_param->value is an output parameter.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - Unrecognised param or param does not support GET
+ * * Return code as defined for param
+ */
+#define ARM_CLA_IOCTL_GET_PARAM \
+ _IOWR(ARM_CLA_IOC_MAGIC, 0x00, struct arm_cla_param)
+
+/**
+ * define ARM_CLA_PARAM_UABI_VERSION - CLA driver UABI version (RO).
+ *
+ * &arm_cla_param->index must be 0.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - index not 0
+ */
+#define ARM_CLA_PARAM_UABI_VERSION 0
+
+/**
+ * define ARM_CLA_PARAM_DEV_NR - Number of attached CLA devices (RO).
+ *
+ * &arm_cla_param->index must be 0.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - index not 0
+ */
+#define ARM_CLA_PARAM_DEV_NR 1
+
+/**
+ * define ARM_CLA_PARAM_DEV_CPU_ID - CPU to which the CLA is attached (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range (0, DEV_NR - 1).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_CPU_ID 2
+
+/**
+ * define ARM_CLA_PARAM_DEV_DOMAIN_ID - Domain to which the CLA belongs (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range (0, DEV_NR - 1).
+ *
+ * A CLA domain contains a set of CLA devices whose accelerators can communicate
+ * with each other. All CLA devices within a CLA domain are atomically assigned
+ * to a single context at a time.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_DOMAIN_ID 3
+
+/**
+ * define ARM_CLA_PARAM_DEV_PGOFF - CLA's ``mmap()`` page offset (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range (0, DEV_NR - 1).
+ *
+ * The page offset at which the selected CLA device's register page is exposed
+ * through ``mmap()``.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_PGOFF 4
+
+/**
+ * define ARM_CLA_PARAM_DEV_AIDR - CLA Architecture Identification Register
+ * (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range (0, DEV_NR - 1).
+ *
+ * The CLA Architecture Identification Register is used to obtain version
+ * information of the CLA Programming model.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_AIDR 5
+
+/**
+ * define ARM_CLA_PARAM_DEV_ACCELS - Mask of CLA's attached accelerators (RO).
+ *
+ * &arm_cla_param->index is the dev_id in range (0, DEV_NR - 1).
+ *
+ * A bitmask describing which accelerator slots are attached on the selected CLA
+ * device. A CLA may have up to 8 attached accelerators, each with an accel_id
+ * in the range (0, 7), each with a corresponding bit in the bitmask.
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id (index) not in range
+ */
+#define ARM_CLA_PARAM_DEV_ACCELS 6
+
+/**
+ * define ARM_CLA_PARAM_ACCEL_IIDR - Accelerator IIDR register (RO).
+ *
+ * &arm_cla_param->index is ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id not in range
+ * * %-EINVAL - accel_id not in range
+ * * %-ENODEV - accel_id not attached to CLA
+ */
+#define ARM_CLA_PARAM_ACCEL_IIDR 7
+
+/**
+ * define ARM_CLA_PARAM_ACCEL_DEVARCH - Accelerator DEVARCH register (RO).
+ *
+ * &arm_cla_param->index is ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id not in range
+ * * %-EINVAL - accel_id not in range
+ * * %-ENODEV - accel_id not attached to CLA
+ */
+#define ARM_CLA_PARAM_ACCEL_DEVARCH 8
+
+/**
+ * define ARM_CLA_PARAM_ACCEL_REVIDR - Accelerator REVIDR register (RO).
+ *
+ * &arm_cla_param->index is ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id).
+ *
+ * Return:
+ * * %0 - OK
+ * * %-EINVAL - dev_id not in range
+ * * %-EINVAL - accel_id not in range
+ * * %-ENODEV - accel_id not attached to CLA
+ */
+#define ARM_CLA_PARAM_ACCEL_REVIDR 9
+
+/**
+ * ARM_CLA_PARAM_ACCEL_INDEX() - encode index for specific accelerator.
+ * @accel_id: accelerator id in the range (0, 7).
+ * @dev_id: CLA device id in the range (0, DEV_NR - 1).
+ *
+ * Return: encoded index for use in &arm_cla_param->index.
+ */
+#define ARM_CLA_PARAM_ACCEL_INDEX(accel_id, dev_id) \
+ (((accel_id) << 24) | (dev_id))
+
+/**
+ * ARM_CLA_PARAM_INDEX_ACCEL() - extract accel_id from index.
+ * @index: Encoded index as returned by ARM_CLA_PARAM_ACCEL_INDEX()
+ *
+ * Return: extracted accel_id.
+ */
+#define ARM_CLA_PARAM_INDEX_ACCEL(index) (((index) >> 24) & 0xff)
+
+/**
+ * ARM_CLA_PARAM_INDEX_DEV() - extract dev_id from index.
+ * @index: Encoded index as returned by ARM_CLA_PARAM_ACCEL_INDEX()
+ *
+ * Return: extracted dev_id.
+ */
+#define ARM_CLA_PARAM_INDEX_DEV(index) ((index) & 0xffffff)
+
+/**
+ * struct arm_cla_param - Get/Set CLA parameters.
+ */
+struct arm_cla_param {
+ /**
+ * @param: Param selector - One of the ARM_CLA_PARAM_* values.
+ */
+ __u64 param;
+
+ /**
+ * @index: Index for params that have multiple instances.
+ */
+ __u64 index;
+
+ /**
+ * @value: Param value.
+ */
+ __u64 value;
+};
+
+#endif /* _UAPI_LINUX_CLA_H */
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 7/8] misc/arm-cla: Manage domain contexts
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
A CLA domain cannot be used by multiple address spaces concurrently
because its devices share a memory translation context and may
communicate without isolation. Represent each user of a domain with a
context keyed by the mm_struct and device file description, and track
contexts in a per-domain hash table.
Add a domain scheduler that queues contexts for assignment and gives
each one a fixed time slice. Before switching, revoke the outgoing
context's mappings from all VMAs for the device file. Then save its
accelerator state and remove its MTC, before installing the incoming
context's MTC and restoring the incoming accelerator state.
Run hardware operations through CPU-bound kthread workers, since each
CLA can only be accessed from its local CPU. Coordinate the per-device
workers with a domain worker so that every CLA in the domain is switched
as a single assignment unit.
Defer context deassignment and destruction when the final mapping
disappears to avoid taking mmap_lock in the wrong order. Keep the mm and
file alive until the context has been removed from hardware and
reclaimed.
Treat a failure on any device as a failure of the whole domain. Stop
further assignments and wake all waiters when a domain becomes unusable.
Co-developed-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
drivers/misc/arm-cla/Makefile | 2 +
drivers/misc/arm-cla/arm-cla.h | 128 ++++++++
drivers/misc/arm-cla/cla-ctx.c | 142 +++++++++
drivers/misc/arm-cla/cla-init.c | 48 +++
drivers/misc/arm-cla/cla-sched.c | 474 ++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 9 +
6 files changed, 803 insertions(+)
create mode 100644 drivers/misc/arm-cla/cla-ctx.c
create mode 100644 drivers/misc/arm-cla/cla-sched.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index df3ad7e4a1b4..a92c3594e5c9 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -2,9 +2,11 @@
arm-cla-y := \
cla-init.o \
+ cla-ctx.o \
cla-mtc.o \
cla-ops.o \
cla-regs.o \
+ cla-sched.o \
cla-topology.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index ffdb82e5e09c..041d4a8d7e91 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -7,9 +7,21 @@
#ifndef _ARM_CLA_H_
#define _ARM_CLA_H_
+#include <linux/atomic.h>
+#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/io.h>
+#include <linux/kref.h>
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/mm.h>
+#include <linux/mm_types.h>
+#include <linux/mutex.h>
+#include <linux/rhashtable-types.h>
+#include <linux/rwsem.h>
#include <linux/types.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
#include "arm-cla-regs.h"
@@ -18,8 +30,21 @@
#define CLA_NUM_DATA_REGS 8
#define CLA_SRSTATE_LEN 8
+/* Quantum of CLA assignment */
+#define CLA_SLICE_MS 100
+
struct cla_domain;
+struct cla_call_on_cpu {
+ int ret;
+ struct {
+ struct cla_ctx *prev_ctx;
+ struct cla_ctx *next_ctx;
+ unsigned int ctx_id;
+ } sched;
+ struct kthread_work switch_ctx;
+};
+
/**
* struct cla_dev - CLA device
*
@@ -36,6 +61,9 @@ struct cla_domain;
* Mutable, only accessed under @lock:
* @lock: Protects the following members.
* @broken: Hardware failure.
+ * @worker: CPU-bound worker to communicate with CLA.
+ * @worker_sem: Serialize running @call against @worker destruction.
+ * @call: Scheduling work.
*/
struct cla_dev {
unsigned int cpu;
@@ -49,22 +77,50 @@ struct cla_dev {
struct mutex lock;
bool broken;
+ struct kthread_worker *worker;
+ struct rw_semaphore worker_sem;
+ struct cla_call_on_cpu call;
};
/**
* struct cla_domain - Collection of cla_dev
*
+ * The whole domain is assigned to a single cla_ctx at a time.
+ *
* Immutable state:
* @id: Domain identifier, from FW or generated.
* @pg_offset: Mmap offset of the first device.
* @nr_devs: Number of devices in the domain.
* @devs: Devices.
+ *
+ * Mutable, only accessed under @lock:
+ * @lock: Protects the following members.
+ * @ctxs: All live contexts, keyed on mm_struct and file ptr.
+ * @queued_ctxs: Queue of contexts waiting for assignment.
+ * @dying_ctxs: Contexts waiting for reclaim.
+ * @worker: Kthread worker to coordinate reassignment.
+ * @reassign: Delayed work that switches contexts with time slicing.
+ * @reclaim: Work to release and free contexts (after reassignment).
+ *
+ * Mutable, some reads outside the lock:
+ * @broken: Hardware failure in any device in the domain.
+ * @assigned_ctx: Context to which domain is currently assigned.
*/
struct cla_domain {
unsigned int id;
unsigned long pg_offset;
unsigned int nr_devs;
struct cla_dev **devs;
+
+ struct mutex lock;
+ bool broken;
+ struct rhashtable ctxs;
+ struct list_head queued_ctxs;
+ struct list_head dying_ctxs;
+ struct cla_ctx *assigned_ctx;
+ struct kthread_worker *worker;
+ struct kthread_delayed_work reassign;
+ struct kthread_delayed_work reclaim;
};
/**
@@ -84,6 +140,46 @@ struct cla_regs {
u64 regstate[];
};
+struct cla_ctx_key {
+ struct mm_struct *mm;
+ struct file *file;
+};
+
+/**
+ * struct cla_ctx - Domain context
+ *
+ * Immutable state:
+ * @domain: The domain of this context.
+ * @key: Key in cla_domain::ctxs hashtable.
+ * @node: Node in cla_domain::ctxs hashtable.
+ *
+ * Mutable, protected by domain::lock:
+ * @refcnt: Current users of this context.
+ * @queue_node: Node in cla_domain::queued_ctxs or
+ * cla_domain::dying_ctxs.
+ * @waitq: Faulting threads sleep until assignment.
+ *
+ * Mutable, protected by domain::lock, some reads outside the lock:
+ * @mapped: Number of VMAs mapping the context.
+ *
+ * Mutable, written only by domain::reassign and dev::switch_ctx:
+ * @regs: State for each device in domain.
+ * @asid: Pinned ASID of live context.
+ */
+struct cla_ctx {
+ struct kref refcnt;
+ struct rhash_head node;
+
+ struct cla_domain *domain;
+ struct cla_ctx_key key;
+
+ refcount_t mapped;
+ struct list_head queue_node;
+ struct cla_regs **regs;
+ wait_queue_head_t waitq;
+ unsigned int asid;
+};
+
extern struct xarray cla_domains;
extern unsigned int cla_nr_domains;
extern struct cla_dev **cla_lut_cpu;
@@ -126,6 +222,38 @@ static inline void cla_reg_write(struct cla_dev *dev, off_t reg, u64 val)
struct cla_domain *cla_dev_domain_get(struct cla_dev *dev);
int cla_domains_finalise(void);
void cla_domains_free(void);
+int cla_domain_sched_init(struct cla_domain *domain);
+void cla_domain_sched_exit(struct cla_domain *domain);
+void cla_domain_set_broken(struct cla_domain *domain);
+struct cla_ctx *cla_domain_lookup_ctx(struct cla_domain *domain,
+ struct mm_struct *mm, struct file *file);
+int cla_domain_insert_ctx(struct cla_domain *domain, struct cla_ctx *ctx);
+void cla_domain_remove_ctx(struct cla_domain *domain, struct cla_ctx *ctx);
+void cla_domain_schedule_reassignment(struct cla_domain *domain,
+ unsigned long ms);
+void cla_domain_schedule_reclaim(struct cla_domain *domain);
+struct cla_ctx *cla_ctx_map(struct cla_domain *domain, struct mm_struct *mm,
+ struct file *file);
+void cla_ctx_unmap(struct cla_domain *domain, struct mm_struct *mm,
+ struct file *file);
+void cla_ctx_free(struct kref *ref);
+
+static inline void cla_ctx_get(struct cla_ctx *ctx)
+{
+ kref_get(&ctx->refcnt);
+}
+
+static inline void cla_ctx_put(struct cla_ctx *ctx)
+{
+ kref_put(&ctx->refcnt, cla_ctx_free);
+}
+
+static inline bool cla_ctx_is_dying(struct cla_ctx *ctx)
+{
+ return refcount_read(&ctx->mapped) == 0;
+}
+
+void cla_dev_switch_ctx(struct kthread_work *work);
int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
int cla_op_reset(struct cla_dev *dev, unsigned int accid);
diff --git a/drivers/misc/arm-cla/cla-ctx.c b/drivers/misc/arm-cla/cla-ctx.c
new file mode 100644
index 000000000000..db2bf5b7f106
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-ctx.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/err.h>
+#include <linux/file.h>
+#include <linux/gfp.h>
+#include <linux/sched/mm.h>
+#include <linux/slab.h>
+
+#include "arm-cla.h"
+
+/**
+ * cla_ctx_map - Map a process and file to a CLA domain context
+ * @domain: CLA domain
+ * @mm: process address space
+ * @file: CLA device file
+ *
+ * Reuse an existing context for the same @mm and @file, or allocate and insert
+ * a new context into @domain. The context mapping count is incremented before
+ * returning.
+ *
+ * Return: the mapped context on success, or an error pointer on failure
+ */
+struct cla_ctx *cla_ctx_map(struct cla_domain *domain, struct mm_struct *mm,
+ struct file *file)
+{
+ struct cla_ctx *ctx;
+ int ret = -ENOMEM;
+
+ mutex_lock(&domain->lock);
+ if (domain->broken) {
+ ret = -EIO;
+ goto err_unlock;
+ }
+
+ ctx = cla_domain_lookup_ctx(domain, mm, file);
+ if (ctx) {
+ refcount_inc(&ctx->mapped);
+ mutex_unlock(&domain->lock);
+ return ctx;
+ }
+
+ ctx = kzalloc_obj(*ctx, GFP_KERNEL_ACCOUNT);
+ if (!ctx)
+ goto err_unlock;
+
+ kref_init(&ctx->refcnt);
+ refcount_set(&ctx->mapped, 1);
+ ctx->domain = domain;
+ ctx->regs = cla_regs_alloc_domain(domain);
+ if (!ctx->regs)
+ goto err_free;
+
+ mmgrab(mm);
+ ctx->key.mm = mm;
+ get_file(file);
+ ctx->key.file = file;
+ INIT_LIST_HEAD(&ctx->queue_node);
+ init_waitqueue_head(&ctx->waitq);
+
+ ret = cla_domain_insert_ctx(domain, ctx);
+ if (ret)
+ goto err_mmdrop;
+ mutex_unlock(&domain->lock);
+
+ return ctx;
+
+err_mmdrop:
+ mmdrop(ctx->key.mm);
+ fput(ctx->key.file);
+ cla_regs_free_domain(domain, ctx->regs);
+err_free:
+ kfree(ctx);
+err_unlock:
+ mutex_unlock(&domain->lock);
+ return ERR_PTR(ret);
+}
+
+/**
+ * cla_ctx_unmap - Unmap a process and file from a CLA domain context
+ * @domain: CLA domain
+ * @mm: process address space
+ * @file: CLA device file
+ *
+ * Drop a context mapping. When the final mapping is removed, wake waiters,
+ * remove the context from @domain, and schedule it for deassignment and
+ * reclaim.
+ */
+void cla_ctx_unmap(struct cla_domain *domain, struct mm_struct *mm,
+ struct file *file)
+{
+ struct cla_ctx *ctx;
+
+ mutex_lock(&domain->lock);
+ ctx = cla_domain_lookup_ctx(domain, mm, file);
+ WARN_ON(!ctx);
+
+ if (!refcount_dec_and_test(&ctx->mapped)) {
+ mutex_unlock(&domain->lock);
+ return;
+ }
+
+ /* Notify waiters that their context is dying. */
+ wake_up_all(&ctx->waitq);
+ cla_domain_remove_ctx(domain, ctx);
+
+ /*
+ * Since we're holding the mmap_lock for writing, and reassignment work
+ * may be waiting to grab it for reading, we cannot wait for
+ * reassignment here. The context holds a reference to mm_count, so the
+ * mm_struct or pgd are not going away until cla_ctx_free().
+ * If the mm is exiting, the accelerators will now access memory through
+ * an empty pgd and fault silently.
+ *
+ * Schedule deassignment followed by reclaim. If reassignment is
+ * currently running, it reschedules itself once it re-takes the lock:
+ * - After deassigning this context, schedule reassignment as usual. If
+ * no more contexts are queued, the following reassignment is a NOP.
+ * - After assigning this context, insert a deassignment before reclaim.
+ */
+ if (domain->assigned_ctx == ctx)
+ cla_domain_schedule_reassignment(domain, 0);
+ list_move(&ctx->queue_node, &domain->dying_ctxs);
+ cla_domain_schedule_reclaim(domain);
+ mutex_unlock(&domain->lock);
+}
+
+/**
+ * cla_ctx_free - Free a CLA domain context
+ * @ref: context reference counter
+ *
+ * Release resources held by a dying context after final reference is dropped.
+ */
+void cla_ctx_free(struct kref *ref)
+{
+ struct cla_ctx *ctx = container_of(ref, struct cla_ctx, refcnt);
+
+ WARN_ON(!cla_ctx_is_dying(ctx));
+ mmdrop(ctx->key.mm);
+ fput(ctx->key.file);
+ cla_regs_free_domain(ctx->domain, ctx->regs);
+ kfree(ctx);
+}
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 0451d97bc95b..5335c55568ad 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -160,6 +160,7 @@ static void cla_dev_reinit(struct cla_dev *dev)
mutex_lock(&dev->lock);
dev->broken = true;
mutex_unlock(&dev->lock);
+ cla_domain_set_broken(dev->domain);
return;
}
@@ -171,6 +172,46 @@ static void cla_dev_reinit(struct cla_dev *dev)
cla_reg_write(dev, CLA_REG_LRESP, 0);
}
+static int cla_dev_worker_init(struct cla_dev *dev, int cpu)
+{
+ struct kthread_worker *worker;
+
+ worker = kthread_run_worker_on_cpu(cpu, 0, "cla-dev-worker/%u");
+ if (IS_ERR(worker))
+ return PTR_ERR(worker);
+
+ mutex_lock(&dev->lock);
+ WARN_ON(dev->worker);
+ dev->worker = worker;
+ mutex_unlock(&dev->lock);
+
+ return 0;
+}
+
+static void cla_dev_worker_destroy(struct cla_dev *dev)
+{
+ struct kthread_worker *worker;
+
+ /*
+ * Mark the worker as NULL, which prevents any new work from being
+ * queued to it. Then destroy it, which will flush any pending work.
+ * worker_sem guarantees lifetime of worker when flushing work in other
+ * paths. We must reinit the work so that work->worker is not dangling
+ * after releasing worker_sem.
+ */
+ mutex_lock(&dev->lock);
+ worker = dev->worker;
+ dev->worker = NULL;
+ mutex_unlock(&dev->lock);
+
+ if (worker) {
+ down_write(&dev->worker_sem);
+ kthread_destroy_worker(worker);
+ kthread_init_work(&dev->call.switch_ctx, cla_dev_switch_ctx);
+ up_write(&dev->worker_sem);
+ }
+}
+
static int cla_dev_setup(unsigned int cpu)
{
int i;
@@ -227,6 +268,10 @@ static int cla_dev_setup(unsigned int cpu)
dev->accelerators);
}
+ ret = cla_dev_worker_init(dev, cpu);
+ if (ret)
+ goto err;
+
return 0;
err:
cla_dev_reinit(dev);
@@ -245,6 +290,7 @@ static int cla_dev_teardown(unsigned int cpu)
if (!dev)
return 0;
+ cla_dev_worker_destroy(dev);
cla_dev_reinit(dev);
return 0;
@@ -289,6 +335,8 @@ static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
dev->dev = parent;
mutex_init(&dev->lock);
+ init_rwsem(&dev->worker_sem);
+ kthread_init_work(&dev->call.switch_ctx, cla_dev_switch_ctx);
/* Attempt to find device domain, or allocate a new one */
dev->domain = cla_dev_domain_get(dev);
diff --git a/drivers/misc/arm-cla/cla-sched.c b/drivers/misc/arm-cla/cla-sched.c
new file mode 100644
index 000000000000..18b54cc386b7
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-sched.c
@@ -0,0 +1,474 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm CLA driver - domain scheduling
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/rhashtable.h>
+#include <linux/mmu_context.h>
+
+#include "arm-cla.h"
+
+static const struct rhashtable_params ctxs_rht_params = {
+ .head_offset = offsetof(struct cla_ctx, node),
+ .key_offset = offsetof(struct cla_ctx, key),
+ .key_len = sizeof(((struct cla_ctx *)0)->key),
+ .automatic_shrinking = true,
+};
+
+static int __cla_domain_switch_ctx(struct cla_domain *domain,
+ struct cla_ctx *prev_ctx,
+ struct cla_ctx *next_ctx)
+{
+ int ret = 0;
+ unsigned int i;
+ struct cla_dev *dev;
+ struct cla_call_on_cpu *call;
+
+ if (!prev_ctx && !next_ctx)
+ return 0;
+
+ for (i = 0; i < domain->nr_devs; i++) {
+ dev = domain->devs[i];
+ call = &dev->call;
+
+ call->sched.prev_ctx = prev_ctx;
+ call->sched.next_ctx = next_ctx;
+ call->sched.ctx_id = i;
+ call->ret = 0;
+
+ mutex_lock(&dev->lock);
+ if (dev->worker && !dev->broken)
+ kthread_queue_work(dev->worker, &call->switch_ctx);
+ mutex_unlock(&dev->lock);
+ }
+
+ for (i = 0; i < domain->nr_devs; i++) {
+ dev = domain->devs[i];
+ call = &dev->call;
+
+ down_read(&dev->worker_sem);
+ kthread_flush_work(&call->switch_ctx);
+ up_read(&dev->worker_sem);
+
+ /*
+ * kthread worker and kthread_flush_work() both take the worker
+ * lock to respectively write and read the current_work,
+ * providing the desired memory ordering for ret.
+ */
+ if (call->ret)
+ ret = call->ret;
+ }
+
+ return ret;
+}
+
+static int cla_domain_switch_ctx(struct cla_domain *domain,
+ struct cla_ctx *prev_ctx,
+ struct cla_ctx *next_ctx)
+{
+ int ret;
+
+ /* Remove prev_ctx from all devices in the domain */
+ ret = __cla_domain_switch_ctx(domain, prev_ctx, NULL);
+ if (ret) {
+ mmgrab(prev_ctx->key.mm);
+ goto err_kill_domain;
+ }
+
+ if (prev_ctx)
+ arm64_mm_context_put(prev_ctx->key.mm);
+ if (next_ctx) {
+ next_ctx->asid = arm64_mm_context_get(next_ctx->key.mm);
+ if (next_ctx->asid == 0) {
+ pr_err("cla: out of pinned ASIDs\n");
+ ret = -ENOSPC;
+ goto err_kill_domain;
+ }
+ }
+
+ /* Install next_ctx into all devices in the domain */
+ ret = __cla_domain_switch_ctx(domain, NULL, next_ctx);
+ if (ret) {
+ mmgrab(next_ctx->key.mm);
+ goto err_kill_domain;
+ }
+ return 0;
+
+err_kill_domain:
+ /* Hardware failure, the domain is dead */
+ cla_domain_set_broken(domain);
+ return ret;
+}
+
+/*
+ * This should only be called within the context of the device's worker, and
+ * so is bound to the cla's local cpu. The worker ensures serialization of
+ * invocations for the same cla. cla_dev_switch_ctx() may be running
+ * concurrently on other cpus for other clas in the same domain.
+ */
+static int __cla_dev_switch_ctx(struct cla_dev *dev)
+{
+ struct cla_ctx *prev_ctx, *next_ctx;
+ unsigned long ctx_id;
+ int ret;
+
+ if (WARN_ON(smp_processor_id() != dev->cpu))
+ return -EINVAL;
+
+ prev_ctx = dev->call.sched.prev_ctx;
+ next_ctx = dev->call.sched.next_ctx;
+ ctx_id = dev->call.sched.ctx_id;
+
+ /* Save state for outgoing ctx. */
+ if (prev_ctx) {
+ /*
+ * Since we're not holding the domain lock during this check, we
+ * may be saving a dying context, but it's only an optimisation.
+ */
+ bool do_save = !cla_ctx_is_dying(prev_ctx);
+
+ ret = cla_regs_switch_out(dev, prev_ctx->regs[ctx_id], do_save);
+ if (ret) {
+ cla_err(dev, "failed to remove regs: %d\n", ret);
+ return ret;
+ }
+
+ ret = cla_mtc_uninstall(dev);
+ if (ret) {
+ cla_err(dev, "failed to remove mm: %d\n", ret);
+ return ret;
+ }
+ }
+
+ /* Restore state for incoming ctx. */
+ if (next_ctx) {
+ ret = cla_mtc_install(dev, next_ctx->key.mm->pgd, next_ctx->asid);
+ if (ret) {
+ cla_err(dev, "failed to setup mm: %d\n", ret);
+ return ret;
+ }
+
+ ret = cla_regs_switch_in(dev, next_ctx->regs[ctx_id]);
+ if (ret) {
+ cla_err(dev, "failed to setup regs: %d\n", ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+void cla_dev_switch_ctx(struct kthread_work *work)
+{
+ struct cla_call_on_cpu *call =
+ container_of(work, struct cla_call_on_cpu, switch_ctx);
+ struct cla_dev *dev = container_of(call, struct cla_dev, call);
+
+ call->ret = __cla_dev_switch_ctx(dev);
+ if (call->ret) {
+ mutex_lock(&dev->lock);
+ dev->broken = true;
+ mutex_unlock(&dev->lock);
+ }
+}
+
+static inline struct cla_ctx *cla_domain_get_next_ctx(struct cla_domain *domain)
+{
+ return list_first_entry_or_null(&domain->queued_ctxs, struct cla_ctx,
+ queue_node);
+}
+
+static inline struct cla_ctx *cla_domain_get_prev_ctx(struct cla_domain *domain,
+ struct cla_ctx *next_ctx)
+{
+ if (domain->assigned_ctx && (next_ctx || cla_ctx_is_dying(domain->assigned_ctx)))
+ return domain->assigned_ctx;
+ return NULL;
+}
+
+static void cla_vma_zap_domain(struct vm_area_struct *vma, struct cla_domain *domain)
+{
+ unsigned long vma_start = vma->vm_pgoff;
+ unsigned long vma_end = vma_start + vma_pages(vma);
+ unsigned long domain_start = domain->pg_offset;
+ unsigned long domain_end = domain_start + domain->nr_devs;
+ unsigned long start = max(vma_start, domain_start);
+ unsigned long end = min(vma_end, domain_end);
+ unsigned long addr;
+ unsigned long size;
+
+ if (start >= end)
+ return;
+
+ addr = vma->vm_start + ((start - vma_start) << PAGE_SHIFT);
+ size = (end - start) << PAGE_SHIFT;
+ zap_special_vma_range(vma, addr, size);
+}
+
+static void cla_domain_zap(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ struct vm_area_struct *vma;
+ struct vma_iterator vmi;
+
+ /*
+ * If the context is dying it has already been unmapped, no need to zap
+ * it here.
+ */
+ if (!ctx || cla_ctx_is_dying(ctx))
+ return;
+
+ /*
+ * Iterate over vmas in prev_ctx's mm, looking for vmas that map
+ * prev_ctx's file. Zap any portions of those vmas that map the domain.
+ */
+ mmap_read_lock(ctx->key.mm);
+ vma_iter_init(&vmi, ctx->key.mm, 0);
+ for_each_vma(vmi, vma) {
+ if (vma->vm_file != ctx->key.file)
+ continue;
+
+ cla_vma_zap_domain(vma, domain);
+ }
+ mmap_read_unlock(ctx->key.mm);
+}
+
+static void cla_domain_reassign(struct kthread_work *work)
+{
+ struct cla_ctx invalid_ctx;
+ struct cla_domain *domain;
+ struct cla_ctx *prev_ctx;
+ struct cla_ctx *next_ctx;
+ int ret;
+
+ domain = container_of(work, struct cla_domain, reassign.work);
+
+ mutex_lock(&domain->lock);
+ if (domain->broken)
+ goto out_unlock;
+
+ /* Figure out prev_ctx and next_ctx. NULL indicates don't [un]assign. */
+ next_ctx = cla_domain_get_next_ctx(domain);
+ prev_ctx = cla_domain_get_prev_ctx(domain, next_ctx);
+
+ if (!prev_ctx && !next_ctx)
+ goto out_unlock;
+
+ /*
+ * We need to grab mmap_lock, but can't do so while holding the domain
+ * lock (since cla_vma_fault() grabs the domain lock while holding
+ * mmap_lock). We are the only thread that modifies assigned_ctx, but
+ * the queue may change and contexts may be killed by cla_ctx_unmap().
+ *
+ * No user is allowed to access CLA while we are reassigning. Disable
+ * assigned_ctx before zapping so cla_vma_fault() won't re-map the
+ * domain during reassignment. A NULL assigned_ctx causes
+ * cla_vma_fault() to immediately schedule a reassignment; use a
+ * reserved context to make it wait.
+ */
+ WRITE_ONCE(domain->assigned_ctx, &invalid_ctx);
+ mutex_unlock(&domain->lock);
+
+ /* Prevent prev_ctx from accessing any device in the domain. */
+ cla_domain_zap(domain, prev_ctx);
+
+ /* Do per-device reassignment work and wait for it to complete. */
+ ret = cla_domain_switch_ctx(domain, prev_ctx, next_ctx);
+ if (ret)
+ next_ctx = NULL;
+
+ mutex_lock(&domain->lock);
+ WRITE_ONCE(domain->assigned_ctx, next_ctx);
+
+ /*
+ * Remove next_ctx from the queue and wake up all waiters so that the
+ * fault handler can map the domain into the newly assigned process.
+ * assigned_ctx must have been updated prior to waking the waiters.
+ *
+ * If next_ctx was killed while we were assigning it, insert a
+ * deassignment before the upcoming reclaim.
+ */
+ if (next_ctx) {
+ if (cla_ctx_is_dying(next_ctx)) {
+ cla_domain_schedule_reassignment(domain, 0);
+ cla_domain_schedule_reclaim(domain);
+ goto out_unlock;
+ }
+ list_del_init(&next_ctx->queue_node);
+ wake_up_all(&next_ctx->waitq);
+ }
+
+ /* If there are more queued ctxs, schedule the next reassignment. */
+ if (!list_empty(&domain->queued_ctxs))
+ cla_domain_schedule_reassignment(domain, CLA_SLICE_MS);
+
+out_unlock:
+ mutex_unlock(&domain->lock);
+}
+
+static void cla_domain_reclaim(struct kthread_work *work)
+{
+ struct cla_domain *domain;
+ struct cla_ctx *ctx, *next;
+
+ domain = container_of(work, struct cla_domain, reclaim.work);
+
+ mutex_lock(&domain->lock);
+ list_for_each_entry_safe(ctx, next, &domain->dying_ctxs, queue_node) {
+ list_del_init(&ctx->queue_node);
+ WARN_ON(domain->assigned_ctx == ctx);
+ cla_ctx_put(ctx);
+ }
+ mutex_unlock(&domain->lock);
+}
+
+/**
+ * cla_domain_sched_init - Initialize scheduling for a CLA domain
+ * @domain: CLA domain
+ *
+ * Initialize domain context table, scheduling queues, worker, and work items.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_domain_sched_init(struct cla_domain *domain)
+{
+ int ret;
+
+ INIT_LIST_HEAD(&domain->queued_ctxs);
+ INIT_LIST_HEAD(&domain->dying_ctxs);
+ WRITE_ONCE(domain->assigned_ctx, NULL);
+
+ ret = rhashtable_init(&domain->ctxs, &ctxs_rht_params);
+ if (ret)
+ return ret;
+
+ domain->worker = kthread_run_worker(0, "cla-domain-worker");
+ if (IS_ERR(domain->worker)) {
+ rhashtable_destroy(&domain->ctxs);
+ return PTR_ERR(domain->worker);
+ }
+ kthread_init_delayed_work(&domain->reassign, cla_domain_reassign);
+ kthread_init_delayed_work(&domain->reclaim, cla_domain_reclaim);
+
+ return 0;
+}
+
+/**
+ * cla_domain_sched_exit - Tear down scheduling for a CLA domain
+ * @domain: CLA domain
+ *
+ * Cancel pending reassignment work, flush reclaim work, destroy the domain
+ * worker, and destroy the context table.
+ */
+void cla_domain_sched_exit(struct cla_domain *domain)
+{
+ WARN_ON(!list_empty(&domain->queued_ctxs));
+ WARN_ON(READ_ONCE(domain->assigned_ctx));
+
+ kthread_cancel_delayed_work_sync(&domain->reassign);
+ /* Since reclaim is always queued with 0 delay, it gets flushed here. */
+ kthread_destroy_worker(domain->worker);
+ rhashtable_destroy(&domain->ctxs);
+}
+
+/**
+ * cla_domain_set_broken - Mark a CLA domain as broken
+ * @domain: CLA domain
+ *
+ * Prevent further reassignment work and wake contexts waiting for assignment.
+ */
+void cla_domain_set_broken(struct cla_domain *domain)
+{
+ struct cla_ctx *ctx;
+
+ mutex_lock(&domain->lock);
+ WRITE_ONCE(domain->broken, true);
+ /* Notify any waiter that their context isn't getting reassigned */
+ list_for_each_entry(ctx, &domain->queued_ctxs, queue_node)
+ wake_up_all(&ctx->waitq);
+ mutex_unlock(&domain->lock);
+}
+
+/**
+ * cla_domain_lookup_ctx - Look up a context in a CLA domain
+ * @domain: CLA domain
+ * @mm: process address space
+ * @file: CLA device file
+ *
+ * The caller must hold the domain lock.
+ *
+ * Return: the matching context, or %NULL if no context was found
+ */
+struct cla_ctx *cla_domain_lookup_ctx(struct cla_domain *domain,
+ struct mm_struct *mm, struct file *file)
+{
+ struct cla_ctx_key key = {
+ .mm = mm,
+ .file = file,
+ };
+
+ lockdep_assert_held(&domain->lock);
+ return rhashtable_lookup_fast(&domain->ctxs, &key, ctxs_rht_params);
+}
+
+/**
+ * cla_domain_insert_ctx - Insert a context into a CLA domain
+ * @domain: CLA domain
+ * @ctx: context to insert
+ *
+ * The caller must hold the domain lock.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_domain_insert_ctx(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ lockdep_assert_held(&domain->lock);
+ return rhashtable_insert_fast(&domain->ctxs, &ctx->node, ctxs_rht_params);
+}
+
+/**
+ * cla_domain_remove_ctx - Remove a context from a CLA domain
+ * @domain: CLA domain
+ * @ctx: context to remove
+ *
+ * The caller must hold the domain lock.
+ */
+void cla_domain_remove_ctx(struct cla_domain *domain, struct cla_ctx *ctx)
+{
+ lockdep_assert_held(&domain->lock);
+ rhashtable_remove_fast(&domain->ctxs, &ctx->node, ctxs_rht_params);
+}
+
+/**
+ * cla_domain_schedule_reassignment - Schedule domain reassignment
+ * @domain: CLA domain
+ * @ms: delay in milliseconds
+ *
+ * The caller must hold the domain lock. Nop if the domain is broken.
+ */
+void cla_domain_schedule_reassignment(struct cla_domain *domain, unsigned long ms)
+{
+ lockdep_assert_held(&domain->lock);
+ if (!domain->broken)
+ kthread_mod_delayed_work(domain->worker, &domain->reassign,
+ msecs_to_jiffies(ms));
+}
+
+/**
+ * cla_domain_schedule_reclaim - Schedule reclaim of dying contexts
+ * @domain: CLA domain
+ *
+ * The caller must hold the domain lock. Reclaim is queued without delay after
+ * any work already pending on the domain worker.
+ */
+void cla_domain_schedule_reclaim(struct cla_domain *domain)
+{
+ lockdep_assert_held(&domain->lock);
+ /*
+ * Reclaim is always queued with 0 delay. It is a delayed work only so
+ * we can easily move the work to the back of the queue (after
+ * reassign). Doing so is tricky with non-delayed work.
+ */
+ kthread_mod_delayed_work(domain->worker, &domain->reclaim, 0);
+}
diff --git a/drivers/misc/arm-cla/cla-topology.c b/drivers/misc/arm-cla/cla-topology.c
index 402c228e197c..ca416b610cb1 100644
--- a/drivers/misc/arm-cla/cla-topology.c
+++ b/drivers/misc/arm-cla/cla-topology.c
@@ -37,6 +37,8 @@ static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int id)
if (!domain)
return ERR_PTR(-ENOMEM);
+ mutex_init(&domain->lock);
+
domain->id = id;
ret = xa_insert(&cla_domains, id, domain, GFP_KERNEL);
if (ret < 0)
@@ -50,8 +52,14 @@ static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int id)
}
domain->devs[0] = dev;
+ ret = cla_domain_sched_init(domain);
+ if (ret)
+ goto err_free_devs;
+
return domain;
+err_free_devs:
+ kfree(domain->devs);
err_free_id:
xa_erase(&cla_domains, domain->id);
err_free_domain:
@@ -155,6 +163,7 @@ int __init cla_domains_finalise(void)
static void cla_domain_free(struct cla_domain *domain)
{
+ cla_domain_sched_exit(domain);
kfree(domain->devs);
xa_erase(&cla_domains, domain->id);
kfree(domain);
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 6/8] misc/arm-cla: Set up memory translation context
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
CLA accelerators use a Memory Translation Context to describe the
page-table and memory-system configuration used for DMA through the CPU
MMU. Set up the kernel-owned MTC during CLA bringup so later context
switches only need to replace the user TTBR0 value.
Populate the MTC from the CPU EL1 configuration, including SCTLR, MAIR,
TCR and TCR2 where present. Configure the accelerator to run as EL0, put
the ASID in TTBR0 and disable TTBR1 translations because the driver only
installs user contexts in the TTBR0 range. When running at EL2, also
program the host HCR state.
Add SETCTX and GETCTX launch helpers for MTC register access. Use the
first available accelerator ID for these operations, since the MTC is
shared by the CLA but the operation still requires a valid accelerator
ID.
Clear the MTC on CPU teardown, and add helpers to install and uninstall
a user translation context by updating TTBR0 with the process page
table, ASID and CnP state.
Co-developed-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
drivers/misc/arm-cla/Makefile | 1 +
drivers/misc/arm-cla/arm-cla.h | 9 +++
drivers/misc/arm-cla/cla-init.c | 10 ++-
drivers/misc/arm-cla/cla-mtc.c | 139 ++++++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-ops.c | 52 ++++++++++++
5 files changed, 209 insertions(+), 2 deletions(-)
create mode 100644 drivers/misc/arm-cla/cla-mtc.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index c1b14155454d..df3ad7e4a1b4 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -2,6 +2,7 @@
arm-cla-y := \
cla-init.o \
+ cla-mtc.o \
cla-ops.o \
cla-regs.o \
cla-topology.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index f1c6d509c3fe..ffdb82e5e09c 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -134,6 +134,10 @@ int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
size_t nregs, u64 *regs);
int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
unsigned int regidx, size_t nregs, u64 *regs);
+int cla_op_setctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs);
+int cla_op_getctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs);
int cla_op_entersr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
int cla_op_exitsr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
@@ -143,4 +147,9 @@ int cla_regs_switch_in(struct cla_dev *dev, struct cla_regs *regs);
struct cla_regs **cla_regs_alloc_domain(struct cla_domain *domain);
void cla_regs_free_domain(struct cla_domain *domain, struct cla_regs **regs);
+int cla_mtc_setup(struct cla_dev *dev);
+int cla_mtc_clear(struct cla_dev *dev);
+int cla_mtc_install(struct cla_dev *dev, pgd_t *pgd, unsigned long asid);
+int cla_mtc_uninstall(struct cla_dev *dev);
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 189ab2a139f1..0451d97bc95b 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -155,7 +155,8 @@ static void cla_dev_reinit(struct cla_dev *dev)
if (broken)
return;
- if (WARN_ON(cla_op_reset_all(dev))) {
+ if (WARN_ON(cla_op_reset_all(dev)) ||
+ WARN_ON(cla_mtc_clear(dev))) {
mutex_lock(&dev->lock);
dev->broken = true;
mutex_unlock(&dev->lock);
@@ -217,9 +218,14 @@ static int cla_dev_setup(unsigned int cpu)
cla_reg_write(dev, CLA_REG_PL0CTRL, plxctrl_val);
- if (dev->accelerators)
+ if (dev->accelerators) {
+ ret = cla_mtc_setup(dev);
+ if (ret)
+ goto err;
+
cla_info(dev, "available accelerators: 0x%02x\n",
dev->accelerators);
+ }
return 0;
err:
diff --git a/drivers/misc/arm-cla/cla-mtc.c b/drivers/misc/arm-cla/cla-mtc.c
new file mode 100644
index 000000000000..0f786d5061da
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-mtc.c
@@ -0,0 +1,139 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - memory translation context
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <asm/kvm_arm.h>
+
+#include "arm-cla.h"
+
+#define cla_kernel_mtc \
+ (cla_kernel_pl == 2 ? CLA_MTC_REGIDX_PL2 : CLA_MTC_REGIDX_PL1)
+
+/**
+ * cla_mtc_setup - initialize the memory translation context
+ * @dev: CLA device
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_setup(struct cla_dev *dev)
+{
+ int ret;
+ u64 tcr;
+ u64 regs[2 * CLA_MTC_PL_SIZE] = {};
+ u64 *kregs = ®s[cla_kernel_mtc];
+ size_t regs_size = cla_kernel_pl * CLA_MTC_PL_SIZE;
+ u64 reserved_ttbr = phys_to_ttbr(page_to_phys(ZERO_PAGE(0)));
+
+ /* The accelerator always runs as EL0 */
+ kregs[CLA_MTC_PSTATE] = FIELD_PREP(CLA_MTC_PSTATE_EL, 0) |
+ FIELD_PREP(CLA_MTC_PSTATE_PAN,
+ system_uses_hw_pan());
+ kregs[CLA_MTC_TTBR0] = reserved_ttbr;
+ kregs[CLA_MTC_TTBR1] = reserved_ttbr;
+ kregs[CLA_MTC_SCTLR] = read_sysreg(sctlr_el1);
+ kregs[CLA_MTC_MAIR] = read_sysreg(mair_el1);
+
+ if (cpus_have_final_cap(ARM64_HAS_TCR2))
+ kregs[CLA_MTC_TCR2] = read_sysreg_s(SYS_TCR2_EL1);
+
+ tcr = read_sysreg(tcr_el1);
+ /*
+ * Put ASID in TTBR0, so that we only have one reg to update when
+ * switching context.
+ */
+ FIELD_MODIFY(TCR_EL1_A1, &tcr, 0);
+ /*
+ * We never map anything in the TTBR1 VA range so explicitly disable
+ * translations via TTBR1.
+ */
+ FIELD_MODIFY(TCR_EL1_EPD1_MASK, &tcr, 1);
+ kregs[CLA_MTC_TCR] = tcr;
+
+ if (cla_kernel_pl == 2) {
+ /* E2H and TGE */
+ regs[CLA_MTC_HCR_EL2] = HCR_HOST_VHE_FLAGS;
+ }
+
+ /*
+ * All accelerators are idle, meaning there can't be any memory
+ * transactions happening. So it is safe to setup the MTC in any order.
+ * i.e. there is no issue with setting SCTLR.M=1 before we have
+ * configured MAIR or TCR2.
+ */
+ ret = cla_op_setctx(dev, CLA_MTC_REGIDX_PL1, regs_size, regs);
+ if (ret) {
+ WARN_ON(cla_mtc_clear(dev));
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * cla_mtc_clear - clear the memory translation context
+ * @dev: CLA device
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_clear(struct cla_dev *dev)
+{
+ /* Don't set the RES1 bit in VTCR_EL2 because in MTC they are RAZ/WI. */
+ u64 regs[2 * CLA_MTC_PL_SIZE] = {};
+ size_t regs_size = cla_kernel_pl * CLA_MTC_PL_SIZE;
+
+ /* SETCTX fails if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ /*
+ * All accelerators are idle, meaning there can't be any memory
+ * transactions happening. So it is safe to clear the MTC in any order.
+ * i.e. there is no issue with clearing TTBRx before setting SCTLR.M=0.
+ */
+ return cla_op_setctx(dev, CLA_MTC_REGIDX_PL1, regs_size, regs);
+}
+
+/**
+ * cla_mtc_install - enable a user translation context
+ * @dev: CLA device
+ * @pgd: page table to install
+ * @asid: ASID to use with the page table
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_install(struct cla_dev *dev, pgd_t *pgd, unsigned long asid)
+{
+ u64 ttbr0;
+
+ /* SETCTX fails if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ ttbr0 = phys_to_ttbr(virt_to_phys(pgd)) |
+ FIELD_PREP(TTBRx_EL1_ASID_MASK, asid);
+
+ if (system_supports_cnp())
+ ttbr0 |= TTBRx_EL1_CnP;
+
+ return cla_op_setctx(dev, cla_kernel_mtc + CLA_MTC_TTBR0, 1, &ttbr0);
+}
+
+/**
+ * cla_mtc_uninstall - disable the user translation context
+ * @dev: CLA device
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int cla_mtc_uninstall(struct cla_dev *dev)
+{
+ u64 ttbr0 = phys_to_ttbr(page_to_phys(ZERO_PAGE(0)));
+
+ /* SETCTX fails if there is no ACCID */
+ if (!dev->accelerators)
+ return 0;
+
+ return cla_op_setctx(dev, cla_kernel_mtc + CLA_MTC_TTBR0, 1, &ttbr0);
+}
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
index 7ddb973927e9..4d3f4f453e0b 100644
--- a/drivers/misc/arm-cla/cla-ops.c
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -180,6 +180,10 @@ static int cla_op_access_reg(struct cla_dev *dev, u8 op,
case CLA_LAUNCH_OP_REGWRITE:
max_regidx = 0x100000000;
break;
+ case CLA_LAUNCH_OP_SETCTX:
+ case CLA_LAUNCH_OP_GETCTX:
+ max_regidx = 0x80;
+ break;
default:
WARN_ON(1);
return -EINVAL;
@@ -246,6 +250,54 @@ int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
accid, regidx, nregs, regs);
}
+/**
+ * cla_op_setctx - launch SETCTX operation.
+ * @dev: CLA device.
+ * @regidx: first MTC register index.
+ * @nregs: number of registers to set. Can be greater than 8 (accessed with
+ * multiple SETCTX operations).
+ * @regs: register content, array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_setctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs)
+{
+ /*
+ * Even though the MTC is shared between all accelerators, accid must be
+ * valid. If no accelerator is attached, the op fails.
+ */
+ if (!dev->accelerators)
+ return -ENODEV;
+
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_SETCTX, CLA_DATA_IN,
+ __ffs(dev->accelerators), regidx, nregs, regs);
+}
+
+/**
+ * cla_op_getctx - launch GETCTX operation.
+ * @dev: CLA device.
+ * @regidx: first MTC register index.
+ * @nregs: number of registers to set. Can be greater than 8 (accessed with
+ * multiple GETCTX operations).
+ * @regs: register content, array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_getctx(struct cla_dev *dev, unsigned int regidx, size_t nregs,
+ u64 *regs)
+{
+ /*
+ * Even though the MTC is shared between all accelerators, accid must be
+ * valid. If no accelerator is attached, the op fails.
+ */
+ if (!dev->accelerators)
+ return -ENODEV;
+
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_GETCTX, CLA_DATA_OUT,
+ __ffs(dev->accelerators), regidx, nregs, regs);
+}
+
/**
* cla_op_entersr - launch ENTERSR operation.
* @dev: CLA device.
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 5/8] misc/arm-cla: Accelerator context save and restore
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
From: Jean-Philippe Brucker <jpb@kernel.org>
Add helpers to save and restore CLA context for later domain switching.
The saved state includes DATA registers, LRESP, per-accelerator SRSTATE
and any implementation-defined internal accelerator state exposed
through the IAS register window.
Use ENTERSR to stop each accelerator and collect SRSTATE, then read the
advertised internal register state with REGREAD. Restore by entering SR
mode from a reset, idle accelerator, writing the saved internal state
with REGWRITE and completing the transition with EXITSR.
Reject accelerators that require SAVE/RESTORE operations during setup,
since that path needs a DMA buffer for state storage. We don't have any
accelerators that support this anyway. Allocate per-device save areas
for all CLAs in a domain, sized from the combined IASSIZE discovered at
setup time.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Makefile | 1 +
drivers/misc/arm-cla/arm-cla.h | 27 ++++
drivers/misc/arm-cla/cla-init.c | 22 +++
drivers/misc/arm-cla/cla-ops.c | 43 ++++++
drivers/misc/arm-cla/cla-regs.c | 263 ++++++++++++++++++++++++++++++++
5 files changed, 356 insertions(+)
create mode 100644 drivers/misc/arm-cla/cla-regs.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index 84ed1a054a21..c1b14155454d 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -3,6 +3,7 @@
arm-cla-y := \
cla-init.o \
cla-ops.o \
+ cla-regs.o \
cla-topology.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 57d0acd01d3a..f1c6d509c3fe 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -30,6 +30,7 @@ struct cla_domain;
* @pfn: Page of registers assigned to user.
* @accelerators: Available accelerators.
* @pg_offset: Mmap offset of this device.
+ * @iassizes: Combined regstate of all accels.
* @domain: The domain this CLA belongs to.
*
* Mutable, only accessed under @lock:
@@ -43,6 +44,7 @@ struct cla_dev {
unsigned long pfn;
u8 accelerators;
unsigned long pg_offset;
+ unsigned long iassizes;
struct cla_domain *domain;
struct mutex lock;
@@ -65,6 +67,23 @@ struct cla_domain {
struct cla_dev **devs;
};
+/**
+ * struct cla_regs - Saved CLA register state
+ *
+ * @data: DATA registers.
+ * @lresp: LRESP register.
+ * @accel_valid: Accelerator state has been saved once.
+ * @srstate: Save/restore state for each accelerator.
+ * @regstate: Internal accelerator state.
+ */
+struct cla_regs {
+ u64 data[CLA_NUM_DATA_REGS];
+ u64 lresp;
+ bool accel_valid;
+ u64 srstate[CLA_NUM_ACC][CLA_SRSTATE_LEN];
+ u64 regstate[];
+};
+
extern struct xarray cla_domains;
extern unsigned int cla_nr_domains;
extern struct cla_dev **cla_lut_cpu;
@@ -115,5 +134,13 @@ int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
size_t nregs, u64 *regs);
int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
unsigned int regidx, size_t nregs, u64 *regs);
+int cla_op_entersr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
+int cla_op_exitsr(struct cla_dev *dev, unsigned int accid, u64 *srstate);
+
+int cla_regs_switch_out(struct cla_dev *dev, struct cla_regs *regs,
+ bool save_regs);
+int cla_regs_switch_in(struct cla_dev *dev, struct cla_regs *regs);
+struct cla_regs **cla_regs_alloc_domain(struct cla_domain *domain);
+void cla_regs_free_domain(struct cla_domain *domain, struct cla_regs **regs);
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index ba7552a2d886..189ab2a139f1 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -81,6 +81,8 @@ static int cla_reset_pmu(struct cla_dev *dev, unsigned int accid)
static int cla_dev_setup_accel(struct cla_dev *dev, unsigned int accid)
{
u64 status;
+ u64 iassize;
+ u64 acap;
int ret;
/*
@@ -106,6 +108,26 @@ static int cla_dev_setup_accel(struct cla_dev *dev, unsigned int accid)
return -EIO;
}
+ /*
+ * We don't support SROP (SAVE and RESTORE ops), only context switching
+ * with REGREAD and REGWRITE. SROP would require finding a DMA buffer
+ * where to save state, ideally in userspace process to avoid kernel
+ * DMA. It's complicated and no implementation needs it at the moment.
+ */
+ ret = cla_op_regread(dev, accid, CLA_REG_ACAP, 1, &acap);
+ if (ret)
+ return ret;
+ if (FIELD_GET(CLA_ACAP_SROP, acap)) {
+ cla_err(dev, "[%u] SROP not supported\n", accid);
+ return 1;
+ }
+
+ ret = cla_op_regread(dev, accid, CLA_REG_IASSIZE, 1, &iassize);
+ if (ret)
+ return ret;
+ if (FIELD_GET(CLA_ACAP_REGSTATE, acap) && iassize)
+ dev->iassizes += iassize;
+
/*
* The following are nice to have, but the accelerator should work
* without them.
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
index f9e5570e89c1..7ddb973927e9 100644
--- a/drivers/misc/arm-cla/cla-ops.c
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -245,3 +245,46 @@ int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
return cla_op_access_reg(dev, CLA_LAUNCH_OP_REGWRITE, CLA_DATA_IN,
accid, regidx, nregs, regs);
}
+
+/**
+ * cla_op_entersr - launch ENTERSR operation.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @srstate: 64-byte save/restore state written by CLA. May be NULL if the
+ * SRSTATE isn't needed.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_entersr(struct cla_dev *dev, unsigned int accid, u64 *srstate)
+{
+ struct cla_launch launch = {
+ .op = CLA_LAUNCH_OP_ENTERSR,
+ .ndata_m1 = 7,
+ .accid = accid,
+ .data_mode = srstate ? CLA_DATA_OUT : CLA_DATA_NONE,
+ .data = srstate,
+ };
+
+ return cla_op_launch(dev, &launch);
+}
+
+/**
+ * cla_op_exitsr - launch EXITSR operation.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @srstate: 64-byte save/restore state written to CLA.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_exitsr(struct cla_dev *dev, unsigned int accid, u64 *srstate)
+{
+ struct cla_launch launch = {
+ .op = CLA_LAUNCH_OP_EXITSR,
+ .ndata_m1 = 7,
+ .accid = accid,
+ .data_mode = CLA_DATA_IN,
+ .data = srstate,
+ };
+
+ return cla_op_launch(dev, &launch);
+}
diff --git a/drivers/misc/arm-cla/cla-regs.c b/drivers/misc/arm-cla/cla-regs.c
new file mode 100644
index 000000000000..7507796cd551
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-regs.c
@@ -0,0 +1,263 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - save/restore accelerator context
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/io.h>
+
+#include "arm-cla.h"
+
+static int cla_regs_save_accel(struct cla_dev *dev, unsigned int accid,
+ struct cla_regs *regs, off_t *regstate_off)
+{
+ int ret;
+ u64 status;
+ size_t regstate_size;
+ u64 *srstate = regs->srstate[accid];
+
+ ret = cla_op_entersr(dev, accid, srstate);
+ if (ret) {
+ /*
+ * Note that we don't expect the accelerator to return an error.
+ * Implementations that don't support context switch cancel the
+ * work and write an error in SRSTATE_STATUS, but don't fail
+ * ENTERSR.
+ */
+ return ret;
+ }
+
+ status = cla_reg_read(dev, CLA_REG_STATUS(accid));
+ if ((status & CLA_STATUS_STATE_MASK) != CLA_STATUS_STATE_SRMODE) {
+ cla_err(dev, "unexpected SR status 0x%llx\n", status);
+ return -EIO;
+ }
+
+ /*
+ * A device that supports SROP fails device probe. Others should not
+ * report SROP here, we don't support it.
+ */
+ WARN_ON(FIELD_GET(CLA_SRSTATE_0_SROP, srstate[0]));
+
+ regstate_size = FIELD_GET(CLA_SRSTATE_0_REGSTATE, srstate[0]);
+ if (WARN_ON(*regstate_off + regstate_size * 8 > dev->iassizes))
+ return -ENOSPC;
+
+ if (regstate_size) {
+ ret = cla_op_regread(dev, accid, CLA_REG_IASn, regstate_size,
+ regs->regstate + *regstate_off);
+ if (ret) {
+ cla_err(dev, "failed to save regstate: %d\n", ret);
+ return ret;
+ }
+ }
+
+ *regstate_off += regstate_size;
+
+ return 0;
+}
+
+/*
+ * Errors are very unlikely, but if they happen the device is left in SRMODE.
+ */
+static int cla_regs_restore_accel(struct cla_dev *dev, unsigned int accid,
+ struct cla_regs *regs, off_t *regstate_off)
+{
+ int ret;
+ u64 status;
+ size_t regstate_size;
+ u64 *srstate = regs->srstate[accid];
+
+ /*
+ * The accelerator was reset and isn't in SRMODE. Later we could support
+ * coming directly from cla_regs_save_accel() in SRMODE, but at the
+ * moment we always need a RESET.
+ */
+ status = cla_reg_read(dev, CLA_REG_STATUS(accid));
+ if ((status & CLA_STATUS_STATE_MASK) != CLA_STATUS_STATE_IDLE) {
+ cla_err(dev, "unexpected status 0x%llx\n", status);
+ return -EIO;
+ }
+
+ ret = cla_op_entersr(dev, accid, NULL);
+ if (ret)
+ return ret;
+
+ WARN_ON(FIELD_GET(CLA_SRSTATE_0_SROP, srstate[0]));
+
+ regstate_size = FIELD_GET(CLA_SRSTATE_0_REGSTATE, srstate[0]);
+ if (WARN_ON(*regstate_off + regstate_size * 8 > dev->iassizes))
+ return -ENOSPC;
+
+ if (regstate_size) {
+ ret = cla_op_regwrite(dev, accid, CLA_REG_IASn, regstate_size,
+ regs->regstate + *regstate_off);
+ if (ret) {
+ cla_err(dev, "failed to restore regstate: %d\n", ret);
+ return ret;
+ }
+ }
+
+ *regstate_off += regstate_size;
+
+ return cla_op_exitsr(dev, accid, srstate);
+}
+
+/**
+ * cla_regs_switch_out - Save CLA context and reset all accelerators
+ * @dev: CLA device
+ * @regs: CLA register state to save
+ * @save_regs: whether to save the DATA and accelerator state
+ *
+ * When this completes, all accelerators are idle. If this fails, some
+ * accelerators may still be running.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_regs_switch_out(struct cla_dev *dev, struct cla_regs *regs,
+ bool save_regs)
+{
+ int i;
+ int ret;
+ unsigned int accid;
+ off_t regstate_off = 0;
+
+ /*
+ * When we interrupt the user process in the middle of launching a
+ * command, we have to wait for LRESP_PENDING to clear before we can
+ * launch a new command or save the DATA registers.
+ */
+ ret = cla_op_wait_lresp(dev, ®s->lresp);
+ if (ret)
+ return ret;
+
+ if (save_regs) {
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ regs->data[i] = cla_reg_read(dev, CLA_REG_DATA(i));
+
+ cla_for_each_accid(dev, accid) {
+ ret = cla_regs_save_accel(dev, accid, regs, ®state_off);
+ if (ret)
+ return ret;
+ }
+
+ regs->accel_valid = true;
+ }
+
+ /*
+ * "If the accelerator is non idle when ENTERSR is launched, then a
+ * RESET operation is required after the internal accelerator state has
+ * been saved"
+ *
+ * However some accelerators keep stale internal state and caches even
+ * after job completion, so always RESET for now.
+ */
+ return cla_op_reset_all(dev);
+}
+
+/**
+ * cla_regs_switch_in - Restore CLA context
+ * @dev: CLA device
+ * @regs: CLA register state to restore
+ *
+ * Restore the DATA and LRESP registers, and accelerator state if one has been
+ * saved. This function is called with all accelerators idle and no trace of
+ * previous work.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_regs_switch_in(struct cla_dev *dev, struct cla_regs *regs)
+{
+ int i;
+ int ret;
+ unsigned int accid;
+ off_t regstate_off = 0;
+
+ cla_for_each_accid(dev, accid) {
+ if (regs->accel_valid) {
+ ret = cla_regs_restore_accel(dev, accid, regs, ®state_off);
+ if (ret)
+ return ret;
+ /*
+ * At this point we must not read STATUS because that
+ * would clear EVENT. Userspace always gets a spurious
+ * EVENT on restore, because we have no way to
+ * save/restore EVENT and userspace missing an event
+ * would be worse than getting a spurious one.
+ */
+ } else {
+ /*
+ * If this context has never been scheduled, then we
+ * just clean the CLA regs. Also clear EVENT by reading
+ * STATUS, to provide a pristine context.
+ */
+ cla_reg_read(dev, CLA_REG_STATUS(accid));
+ }
+ }
+
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), regs->data[i]);
+ cla_reg_write(dev, CLA_REG_LRESP, regs->lresp);
+
+ return 0;
+}
+
+/**
+ * cla_regs_alloc_domain - Allocate register state for a CLA domain
+ * @domain: CLA domain
+ *
+ * Allocate register state to save and restore every device in @domain.
+ *
+ * Return: an array of register state pointers on success, %NULL on failure.
+ */
+struct cla_regs **cla_regs_alloc_domain(struct cla_domain *domain)
+{
+ int i;
+ size_t size;
+ struct cla_regs **regs_ptrs;
+
+ regs_ptrs = kmalloc_objs(*regs_ptrs, domain->nr_devs,
+ GFP_KERNEL_ACCOUNT);
+ if (!regs_ptrs)
+ return NULL;
+
+ for (i = 0; i < domain->nr_devs; i++) {
+ struct cla_regs *regs;
+ struct cla_dev *dev = domain->devs[i];
+
+ /*
+ * The regs structures are only ever accessed from the CLA
+ * device's CPU, so try to allocate them on the right NUMA node
+ */
+ size = sizeof(*regs) + dev->iassizes;
+ regs = kvzalloc_node(size, GFP_KERNEL_ACCOUNT,
+ cpu_to_node(dev->cpu));
+ if (!regs)
+ goto err_free;
+
+ regs_ptrs[i] = regs;
+ }
+
+ return regs_ptrs;
+
+err_free:
+ for (i--; i >= 0; i--)
+ kvfree(regs_ptrs[i]);
+ kfree(regs_ptrs);
+ return NULL;
+}
+
+/**
+ * cla_regs_free_domain - Free register state for a CLA domain
+ * @domain: CLA domain
+ * @regs: array of register state pointers to free
+ */
+void cla_regs_free_domain(struct cla_domain *domain, struct cla_regs **regs)
+{
+ int i;
+
+ for (i = 0; i < domain->nr_devs; i++)
+ kvfree(regs[i]);
+ kfree(regs);
+}
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 4/8] misc/arm-cla: Initialize devices on CPU bringup
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
From: Jean-Philippe Brucker <jpb@kernel.org>
CLA registers are only accessible from the CPU local to the CLA, so
perform device initialization from the CPU hotplug online callback.
Clear the DATA registers, reset each attached accelerator and verify
that it returns to the expected idle state. Record the set of usable
accelerators and expose only those through PL0CTRL, and through PL1CTRL
when the kernel is running at EL2.
Reset optional timestamp and PMU controls where available so userspace
starts from a predictable state, but do not fail probing if those
optional controls cannot be programmed.
On CPU teardown, reset all known accelerators and mark the device broken
if reset fails, since the hotplug teardown path cannot safely fail.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/arm-cla.h | 14 +++
drivers/misc/arm-cla/cla-init.c | 178 ++++++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-ops.c | 19 ++++
3 files changed, 211 insertions(+)
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 1853daa200e4..57d0acd01d3a 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -28,16 +28,25 @@ struct cla_domain;
* @regs: Registers accessed by the kernel.
* @dev: The platform device.
* @pfn: Page of registers assigned to user.
+ * @accelerators: Available accelerators.
* @pg_offset: Mmap offset of this device.
* @domain: The domain this CLA belongs to.
+ *
+ * Mutable, only accessed under @lock:
+ * @lock: Protects the following members.
+ * @broken: Hardware failure.
*/
struct cla_dev {
unsigned int cpu;
void __iomem *regs;
struct device *dev;
unsigned long pfn;
+ u8 accelerators;
unsigned long pg_offset;
struct cla_domain *domain;
+
+ struct mutex lock;
+ bool broken;
};
/**
@@ -62,6 +71,10 @@ extern struct cla_dev **cla_lut_cpu;
extern struct cla_dev **cla_lut_pg;
extern unsigned int cla_nr_devs;
+#define cla_for_each_accid(dev, accid) \
+ for ((accid) = 0; (accid) < CLA_NUM_ACC; (accid)++) \
+ for_each_if((dev)->accelerators & BIT(accid))
+
#define cla_dbg(dev, fmt, ...) \
dev_dbg((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
#define cla_info(dev, fmt, ...) \
@@ -97,6 +110,7 @@ void cla_domains_free(void);
int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
int cla_op_reset(struct cla_dev *dev, unsigned int accid);
+int cla_op_reset_all(struct cla_dev *dev);
int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
size_t nregs, u64 *regs);
int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 731ca08c9b79..ba7552a2d886 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -17,18 +17,192 @@
static int cla_cpuhp_state = -1;
+static int cla_reset_ts(struct cla_dev *dev, unsigned int accid)
+{
+ int ret;
+ u64 reg;
+
+ ret = cla_op_regread(dev, accid, CLA_REG_ACAP, 1, ®);
+ if (ret)
+ return ret;
+ if (!FIELD_GET(CLA_ACAP_TS, reg))
+ return 0;
+
+ /*
+ * Disable TS control from userspace, provide TS from CNTP. If we do
+ * have to provide a timer to userspace or a virtual offset to a guest,
+ * we'll need to make sure we have access to both TSCTRLOWNER and
+ * TSOFFOWNER. For now best effort.
+ */
+ reg = FIELD_PREP(CLA_TSCTRLOWNER_PL, cla_kernel_pl);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSCTRLOWNER, 1, ®);
+ if (!ret) {
+ reg = FIELD_PREP(CLA_TSCTRL_TS, CLA_TSCTRL_PHYSICAL);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSCTRL, 1, ®);
+ if (ret)
+ return ret;
+ }
+
+ reg = FIELD_PREP(CLA_TSOFFOWNER_PL, cla_kernel_pl);
+ ret = cla_op_regread(dev, accid, CLA_REG_TSOFFOWNER, 1, ®);
+ if (!ret) {
+ reg = 0;
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSVOFF, 1, ®);
+ if (ret)
+ return ret;
+ ret = cla_op_regwrite(dev, accid, CLA_REG_TSPOFF, 1, ®);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+static int cla_reset_pmu(struct cla_dev *dev, unsigned int accid)
+{
+ int ret;
+ u64 reg;
+
+ /* Disable PMU access */
+ reg = FIELD_PREP(CLA_PMUOWNER_PL, cla_kernel_pl);
+ ret = cla_op_regwrite(dev, accid, CLA_REG_PMUOWNER, 1, ®);
+ if (!ret) {
+ reg = 0;
+ ret = cla_op_regwrite(dev, accid, CLA_REG_PMURESET, 1, ®);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+/*
+ * Return: 0 on success, 1 if the accelerator is not attached or not usable, or
+ * an error
+ */
+static int cla_dev_setup_accel(struct cla_dev *dev, unsigned int accid)
+{
+ u64 status;
+ int ret;
+
+ /*
+ * Probe and reset. Return 1 if no accelerator is attached, happy days.
+ * If the accelerator is unavailable (masked by higher PL with PLxCTRL),
+ * return an error. Individual accelerators cannot be owned by a higher
+ * PL, since the MTC is shared between all accelerators attached to this
+ * CLA.
+ *
+ * Some accelerators will be masked due to returning 1 further down this
+ * function. If we end up with no dev->accelerators because of that we
+ * won't setup the MTC, but as long as this reset succeeds, the
+ * accelerator is not issuing memory transactions.
+ */
+ ret = cla_op_reset(dev, accid);
+ if (ret)
+ return ret;
+
+ status = cla_reg_read(dev, CLA_REG_STATUS(accid));
+ if ((status & CLA_STATUS_STATE_MASK) != CLA_STATUS_STATE_IDLE) {
+ cla_err(dev, "unexpected status 0x%llx for accelerator %d\n",
+ status, accid);
+ return -EIO;
+ }
+
+ /*
+ * The following are nice to have, but the accelerator should work
+ * without them.
+ */
+ ret = cla_reset_ts(dev, accid);
+ if (ret)
+ cla_err(dev, "[%u] could not reset TS: %d\n", accid, ret);
+
+ ret = cla_reset_pmu(dev, accid);
+ if (ret)
+ cla_err(dev, "[%u] could not reset PMU: %d\n", accid, ret);
+
+ return 0;
+}
+
+/* Clean the device before releasing it */
+static void cla_dev_reinit(struct cla_dev *dev)
+{
+ int i;
+ bool broken;
+
+ mutex_lock(&dev->lock);
+ broken = dev->broken;
+ mutex_unlock(&dev->lock);
+ if (broken)
+ return;
+
+ if (WARN_ON(cla_op_reset_all(dev))) {
+ mutex_lock(&dev->lock);
+ dev->broken = true;
+ mutex_unlock(&dev->lock);
+ return;
+ }
+
+ if (is_kernel_in_hyp_mode())
+ cla_reg_write(dev, CLA_REG_PL1CTRL, ~0ULL);
+ cla_reg_write(dev, CLA_REG_PL0CTRL, ~0ULL);
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), 0);
+ cla_reg_write(dev, CLA_REG_LRESP, 0);
+}
+
static int cla_dev_setup(unsigned int cpu)
{
+ int i;
+ int ret;
+ bool broken;
+ unsigned int accid;
struct cla_dev *dev;
+ u64 plxctrl_val = 0;
dev = cla_lut_cpu[cpu];
if (!dev)
return 0;
+ mutex_lock(&dev->lock);
+ broken = dev->broken;
+ mutex_unlock(&dev->lock);
+ if (broken)
+ return 0;
+
if (WARN_ON(smp_processor_id() != cpu || dev->cpu != cpu))
return -EINVAL;
+ /* Clear DATA and LRESP_DATANZ */
+ for (i = 0; i < CLA_NUM_DATA_REGS; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), 0);
+
+ /*
+ * Reset all accelerators. We restrict PLxCTRL to the accelerators that
+ * are attached and well behaved.
+ */
+ for (accid = 0; accid < CLA_NUM_ACC; accid++) {
+ ret = cla_dev_setup_accel(dev, accid);
+ if (ret > 0)
+ continue;
+ else if (ret < 0)
+ goto err;
+
+ dev->accelerators |= (1 << accid);
+ plxctrl_val |= CLA_PLxCTRL_PREP(accid,
+ FIELD_PREP(CLA_PLxCTRL_AVAIL, 1));
+ }
+
+ if (is_kernel_in_hyp_mode())
+ cla_reg_write(dev, CLA_REG_PL1CTRL, plxctrl_val);
+
+ cla_reg_write(dev, CLA_REG_PL0CTRL, plxctrl_val);
+
+ if (dev->accelerators)
+ cla_info(dev, "available accelerators: 0x%02x\n",
+ dev->accelerators);
+
return 0;
+err:
+ cla_dev_reinit(dev);
+ return ret;
}
static int cla_dev_teardown(unsigned int cpu)
@@ -43,6 +217,8 @@ static int cla_dev_teardown(unsigned int cpu)
if (!dev)
return 0;
+ cla_dev_reinit(dev);
+
return 0;
}
@@ -84,6 +260,8 @@ static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
dev->cpu = cpu;
dev->dev = parent;
+ mutex_init(&dev->lock);
+
/* Attempt to find device domain, or allocate a new one */
dev->domain = cla_dev_domain_get(dev);
if (IS_ERR(dev->domain))
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
index d594344a2eb4..f9e5570e89c1 100644
--- a/drivers/misc/arm-cla/cla-ops.c
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -143,6 +143,25 @@ int cla_op_reset(struct cla_dev *dev, unsigned int accid)
return ret;
}
+/**
+ * cla_op_reset_all - Reset all attached accelerators
+ * @dev: CLA device.
+ *
+ * Return: 0 on success, or an error
+ */
+int cla_op_reset_all(struct cla_dev *dev)
+{
+ int ret;
+ unsigned int accid;
+
+ cla_for_each_accid(dev, accid) {
+ ret = cla_op_reset(dev, accid);
+ if (ret)
+ return ret < 0 ? ret : -ENODEV;
+ }
+ return 0;
+}
+
static int cla_op_access_reg(struct cla_dev *dev, u8 op,
enum cla_launch_data_mode data_mode,
unsigned int accid, unsigned int regidx,
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 3/8] misc/arm-cla: Probe firmware-described devices
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
From: Jean-Philippe Brucker <jpb@kernel.org>
Discover CLA devices from firmware and instantiate the driver with
platform_driver_probe(), since the set of devices is fixed at boot. Map
the MMIO frame, select the kernel-accessible privilege-level frame and
associate each CLA with the CPU named by firmware.
Add the topology code used to group CLAs into domains. Domains represent
the smallest unit that can be assigned independently, because CLAs in
the same domain cannot be isolated from each other. Build lookup tables
indexed by CPU and by mmap page offset for later fast access.
Register CPU hotplug callbacks so that hardware setup and teardown can
run on the CPU local to each CLA, which is the only CPU allowed to
access that CLA's registers.
Co-developed-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Makefile | 3 +-
drivers/misc/arm-cla/arm-cla.h | 46 ++++++
drivers/misc/arm-cla/cla-init.c | 211 ++++++++++++++++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 178 +++++++++++++++++++++++
4 files changed, 437 insertions(+), 1 deletion(-)
create mode 100644 drivers/misc/arm-cla/cla-topology.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index 3fa8567c8b3e..84ed1a054a21 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -2,6 +2,7 @@
arm-cla-y := \
cla-init.o \
- cla-ops.o
+ cla-ops.o \
+ cla-topology.o
obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index 9294b71929f1..1853daa200e4 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -18,6 +18,8 @@
#define CLA_NUM_DATA_REGS 8
#define CLA_SRSTATE_LEN 8
+struct cla_domain;
+
/**
* struct cla_dev - CLA device
*
@@ -25,13 +27,41 @@
* @cpu: The CPU this CLA is attached to.
* @regs: Registers accessed by the kernel.
* @dev: The platform device.
+ * @pfn: Page of registers assigned to user.
+ * @pg_offset: Mmap offset of this device.
+ * @domain: The domain this CLA belongs to.
*/
struct cla_dev {
unsigned int cpu;
void __iomem *regs;
struct device *dev;
+ unsigned long pfn;
+ unsigned long pg_offset;
+ struct cla_domain *domain;
+};
+
+/**
+ * struct cla_domain - Collection of cla_dev
+ *
+ * Immutable state:
+ * @id: Domain identifier, from FW or generated.
+ * @pg_offset: Mmap offset of the first device.
+ * @nr_devs: Number of devices in the domain.
+ * @devs: Devices.
+ */
+struct cla_domain {
+ unsigned int id;
+ unsigned long pg_offset;
+ unsigned int nr_devs;
+ struct cla_dev **devs;
};
+extern struct xarray cla_domains;
+extern unsigned int cla_nr_domains;
+extern struct cla_dev **cla_lut_cpu;
+extern struct cla_dev **cla_lut_pg;
+extern unsigned int cla_nr_devs;
+
#define cla_dbg(dev, fmt, ...) \
dev_dbg((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
#define cla_info(dev, fmt, ...) \
@@ -39,6 +69,13 @@ struct cla_dev {
#define cla_err(dev, fmt, ...) \
dev_err((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
+#define CLA_REG_SIZE SZ_64K
+#define CLA_FRAME_SIZE (4 * CLA_REG_SIZE)
+
+/* Return the registers corresponding to this privilege level */
+#define cla_get_regs(base, pl) \
+ ((typeof(base))((uintptr_t)(base) + (pl) * CLA_REG_SIZE))
+
static inline u64 cla_reg_read(struct cla_dev *dev, off_t reg)
{
return readq_relaxed(dev->regs + reg);
@@ -49,6 +86,15 @@ static inline void cla_reg_write(struct cla_dev *dev, off_t reg, u64 val)
return writeq_relaxed(val, dev->regs + reg);
}
+/*
+ * If we're at EL2, use PL2. If we're a guest or nVHE host, use PL1.
+ */
+#define cla_kernel_pl (is_kernel_in_hyp_mode() ? 2 : 1)
+
+struct cla_domain *cla_dev_domain_get(struct cla_dev *dev);
+int cla_domains_finalise(void);
+void cla_domains_free(void);
+
int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
int cla_op_reset(struct cla_dev *dev, unsigned int accid);
int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
index 3d1f47592842..731ca08c9b79 100644
--- a/drivers/misc/arm-cla/cla-init.c
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -5,7 +5,218 @@
* Copyright 2026 Arm Limited.
*/
+#include <linux/cpuhotplug.h>
#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/smp.h>
+
+#include <asm/virt.h>
+
+#include "arm-cla.h"
+
+static int cla_cpuhp_state = -1;
+
+static int cla_dev_setup(unsigned int cpu)
+{
+ struct cla_dev *dev;
+
+ dev = cla_lut_cpu[cpu];
+ if (!dev)
+ return 0;
+
+ if (WARN_ON(smp_processor_id() != cpu || dev->cpu != cpu))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int cla_dev_teardown(unsigned int cpu)
+{
+ struct cla_dev *dev;
+
+ /*
+ * Careful what we return here, the teardown path isn't really allowed
+ * to fail (BUG_ON in kernel/cpu.c)
+ */
+ dev = cla_lut_cpu[cpu];
+ if (!dev)
+ return 0;
+
+ return 0;
+}
+
+static int cla_of_to_cpu(struct device_node *of_node)
+{
+ int cpu;
+ int ret;
+ u32 cpu_phandle;
+ struct device_node *cpu_node;
+
+ if (!of_node)
+ return -ENODEV;
+
+ ret = of_property_read_u32(of_node, "cpu", &cpu_phandle);
+ if (WARN_ON(ret))
+ return -EINVAL;
+
+ cpu_node = of_find_node_by_phandle(cpu_phandle);
+ if (WARN_ON(!cpu_node))
+ return -EINVAL;
+
+ cpu = of_cpu_node_to_id(cpu_node);
+ of_node_put(cpu_node);
+
+ return cpu;
+}
+
+static struct cla_dev *cla_dev_alloc(struct device *parent, int cpu,
+ void __iomem *regs, phys_addr_t base)
+{
+ struct cla_dev *dev;
+
+ dev = devm_kzalloc(parent, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return ERR_PTR(-ENOMEM);
+
+ dev->pfn = __phys_to_pfn(base);
+ dev->regs = cla_get_regs(regs, cla_kernel_pl);
+ dev->cpu = cpu;
+ dev->dev = parent;
+
+ /* Attempt to find device domain, or allocate a new one */
+ dev->domain = cla_dev_domain_get(dev);
+ if (IS_ERR(dev->domain))
+ return ERR_CAST(dev->domain);
+
+ cla_nr_devs++;
+
+ return dev;
+}
+
+static int cla_probe(struct platform_device *pdev)
+{
+ int cpu;
+ void __iomem *reg;
+ size_t reg_size;
+ struct cla_dev *dev;
+ struct resource *res;
+
+ /*
+ * TODO: the firmware maps this as well to access PL3, and the guest
+ * will map PL1 and PL0. Avoid TLB attr mismatches by only mapping what
+ * we need.
+ */
+ reg = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
+ if (IS_ERR(reg)) {
+ dev_err(&pdev->dev, "could not map CLA registers\n");
+ return PTR_ERR(reg);
+ }
+
+ if (!IS_ALIGNED(res->start, SZ_256K)) {
+ dev_err(&pdev->dev, "invalid CLA registers alignment\n");
+ return -EINVAL;
+ }
+
+ reg_size = resource_size(res);
+ if (reg_size <= CLA_FRAME_SIZE) {
+ /* A single CLA. We need information about its CPU. */
+ cpu = cla_of_to_cpu(pdev->dev.of_node);
+ if (cpu < 0)
+ return cpu;
+
+ /*
+ * As a guest we may not get PL3 or PL2. Tolerate CLAs smaller
+ * than 4*regs.
+ */
+ if (reg_size < (cla_kernel_pl + 1) * CLA_REG_SIZE)
+ return -ENXIO;
+
+ dev = cla_dev_alloc(&pdev->dev, cpu, reg, res->start);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
+
+ dev_dbg(&pdev->dev, "CLA found %pa size 0x%llx\n", &res->start,
+ resource_size(res));
+
+ } else {
+ dev_err(&pdev->dev, "unexpected CLA registers size\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct of_device_id cla_of_match[] = {
+ {.compatible = "arm,cla",},
+ {},
+};
+MODULE_DEVICE_TABLE(of, cla_of_match);
+
+static struct platform_driver cla_driver = {
+ .driver = {
+ .name = "arm-cla",
+ .of_match_table = cla_of_match,
+ },
+ .probe = cla_probe,
+};
+
+static int __init cla_module_init(void)
+{
+ int ret;
+
+ /*
+ * CPUs may be hotplugged, but all CLAs are described by firmware so the
+ * probe can be synchronous. This only sets up the resources, and CPUHP
+ * callbacks will do the actual peeking and poking.
+ *
+ * This returns an error when no CLA is present.
+ */
+ ret = platform_driver_probe(&cla_driver, cla_probe);
+ if (ret) {
+ if (ret != -ENODEV)
+ pr_err("arm-cla: probe failed with %d\n", ret);
+ /* Some domains may have been created during probe */
+ goto err_domains_free;
+ }
+
+ ret = cla_domains_finalise();
+ if (ret) {
+ pr_err("arm-cla: failed to finalise domains: %d", ret);
+ goto err_domains_free;
+ }
+
+ /*
+ * Each CPU initializes their own CLA. CPUHP uses a pair of smp_mb()
+ * when calling the startup callback, ensuring that cla_dev_setup()
+ * reads fully initialized cla_lut_cpu and cla_dev structures.
+ */
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "arm-cla",
+ cla_dev_setup, cla_dev_teardown);
+ if (ret < 0) {
+ pr_err("arm-cla: failed to setup cpuhp: %d", ret);
+ goto err_driver_unregister;
+ }
+ cla_cpuhp_state = ret;
+
+ return 0;
+
+err_driver_unregister:
+ platform_driver_unregister(&cla_driver);
+err_domains_free:
+ cla_domains_free();
+ return ret;
+}
+
+static void __exit cla_module_exit(void)
+{
+ cpuhp_remove_state(cla_cpuhp_state);
+ platform_driver_unregister(&cla_driver);
+ cla_domains_free();
+}
+
+module_init(cla_module_init);
+module_exit(cla_module_exit);
MODULE_DESCRIPTION("Arm Core Local Accelerator");
MODULE_AUTHOR("Arm Limited");
diff --git a/drivers/misc/arm-cla/cla-topology.c b/drivers/misc/arm-cla/cla-topology.c
new file mode 100644
index 000000000000..402c228e197c
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-topology.c
@@ -0,0 +1,178 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Arm CLA driver - device topology initialization
+ *
+ * A CLA domain is a group of devices that work together and cannot be isolated
+ * from each other. They are owned by a single user at a time.
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/device.h>
+#include <linux/errno.h>
+#include <linux/gfp.h>
+#include <linux/kernel.h>
+#include <linux/of.h>
+
+#include "arm-cla.h"
+
+DEFINE_XARRAY_ALLOC(cla_domains);
+unsigned int cla_nr_domains;
+
+/*
+ * Some CPUs may not have a CLA. So cla_lut_cpu (indexed by CPU) may be sparse,
+ * but cla_lut_pg (indexed by page offset in mmap'ed file) is a contiguous array
+ * of all devices, of size cla_nr_devs, sorted by domain.
+ */
+struct cla_dev **cla_lut_cpu;
+struct cla_dev **cla_lut_pg;
+unsigned int cla_nr_devs;
+
+static struct cla_domain *cla_domain_alloc(struct cla_dev *dev, unsigned int id)
+{
+ int ret;
+ struct cla_domain *domain;
+
+ domain = kzalloc_obj(*domain);
+ if (!domain)
+ return ERR_PTR(-ENOMEM);
+
+ domain->id = id;
+ ret = xa_insert(&cla_domains, id, domain, GFP_KERNEL);
+ if (ret < 0)
+ goto err_free_domain;
+
+ domain->nr_devs = 1;
+ domain->devs = kzalloc_obj(*domain->devs);
+ if (!domain->devs) {
+ ret = -ENOMEM;
+ goto err_free_id;
+ }
+ domain->devs[0] = dev;
+
+ return domain;
+
+err_free_id:
+ xa_erase(&cla_domains, domain->id);
+err_free_domain:
+ kfree(domain);
+ return ERR_PTR(ret);
+}
+
+/**
+ * cla_dev_domain_get - get or create a CLA domain for a device
+ * @dev: CLA device
+ *
+ * Return: CLA domain pointer on success, or an ERR_PTR() on failure.
+ */
+struct cla_domain *cla_dev_domain_get(struct cla_dev *dev)
+{
+ int ret;
+ unsigned int domain_id;
+ struct cla_domain *domain;
+
+ /* Domain ID is provided by firmware */
+ ret = of_property_read_u32(dev->dev->of_node, "domain", &domain_id);
+ if (WARN_ON(ret))
+ return ERR_PTR(-EINVAL);
+
+ domain = xa_load(&cla_domains, domain_id);
+ if (domain) {
+ domain->nr_devs++;
+ domain->devs = krealloc_array(domain->devs, domain->nr_devs,
+ sizeof(*domain->devs), GFP_KERNEL);
+ if (!domain->devs)
+ return ERR_PTR(-ENOMEM);
+ domain->devs[domain->nr_devs - 1] = dev;
+ return domain;
+ }
+
+ domain = cla_domain_alloc(dev, domain_id);
+ if (IS_ERR(domain))
+ return domain;
+
+ cla_nr_domains = max(domain_id + 1, cla_nr_domains);
+
+ return domain;
+}
+
+/**
+ * cla_domains_finalise - build CLA device lookup tables
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+int __init cla_domains_finalise(void)
+{
+ int ret = -ENOMEM;
+ unsigned int i, j;
+ unsigned int pg_offset = 0;
+
+ cla_lut_cpu = kzalloc_objs(*cla_lut_cpu, nr_cpu_ids);
+ if (!cla_lut_cpu)
+ goto err_free;
+
+ cla_lut_pg = kzalloc_objs(*cla_lut_pg, cla_nr_devs);
+ if (!cla_lut_pg)
+ goto err_free;
+
+ ret = -EINVAL;
+
+ /* Populate the lookup tables. */
+ for (i = 0; i < cla_nr_domains; i++) {
+ struct cla_domain *domain = xa_load(&cla_domains, i);
+
+ /* The user API requires sequential domain IDs */
+ if (WARN_ON(!domain))
+ goto err_free;
+
+ domain->pg_offset = pg_offset;
+
+ for (j = 0; j < domain->nr_devs; j++) {
+ struct cla_dev *dev = domain->devs[j];
+
+ if (WARN_ON(dev->cpu >= nr_cpu_ids) ||
+ WARN_ON(pg_offset >= cla_nr_devs))
+ goto err_free;
+ WARN_ON(cla_lut_cpu[dev->cpu]);
+ WARN_ON(cla_lut_pg[pg_offset]);
+
+ cla_lut_cpu[dev->cpu] = dev;
+ cla_lut_pg[pg_offset] = dev;
+ dev->pg_offset = pg_offset;
+ pg_offset++;
+ }
+ }
+
+ return 0;
+
+err_free:
+ kfree(cla_lut_cpu);
+ kfree(cla_lut_pg);
+ cla_lut_cpu = NULL;
+ cla_lut_pg = NULL;
+ return ret;
+}
+
+static void cla_domain_free(struct cla_domain *domain)
+{
+ kfree(domain->devs);
+ xa_erase(&cla_domains, domain->id);
+ kfree(domain);
+}
+
+/**
+ * cla_domains_free - free CLA domains and lookup tables
+ */
+void cla_domains_free(void)
+{
+ unsigned long id;
+ struct cla_domain *domain;
+
+ kfree(cla_lut_cpu);
+ kfree(cla_lut_pg);
+ cla_lut_cpu = NULL;
+ cla_lut_pg = NULL;
+ xa_for_each(&cla_domains, id, domain)
+ cla_domain_free(domain);
+ cla_nr_domains = 0;
+}
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 2/8] misc/arm-cla: Add launch operation helpers
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
From: Jean-Philippe Brucker <jpb@kernel.org>
CLA commands are issued by writing optional payload registers,
programming the LAUNCH register and polling LRESP until the hardware
accepts or rejects the operation.
Add a common launch helper that performs this sequence on the CLA's
local CPU, waits for LRESP completion and translates launch response
codes into Linux errors.
Build accelerator reset and register read and write support on top of
it. The register read and write helpers split larger accesses into
multiple launch operations when an access crosses an eight-register
window.
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
drivers/misc/arm-cla/Makefile | 6 +
drivers/misc/arm-cla/arm-cla.h | 21 +++
drivers/misc/arm-cla/cla-init.c | 12 ++
drivers/misc/arm-cla/cla-ops.c | 228 ++++++++++++++++++++++++++++++++
4 files changed, 267 insertions(+)
create mode 100644 drivers/misc/arm-cla/cla-init.c
create mode 100644 drivers/misc/arm-cla/cla-ops.c
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
index a4e40e534e6a..3fa8567c8b3e 100644
--- a/drivers/misc/arm-cla/Makefile
+++ b/drivers/misc/arm-cla/Makefile
@@ -1 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
+
+arm-cla-y := \
+ cla-init.o \
+ cla-ops.o
+
+obj-$(CONFIG_ARM_CLA) += arm-cla.o
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
index f265d7b60268..9294b71929f1 100644
--- a/drivers/misc/arm-cla/arm-cla.h
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -7,6 +7,8 @@
#ifndef _ARM_CLA_H_
#define _ARM_CLA_H_
+#include <linux/device.h>
+#include <linux/io.h>
#include <linux/types.h>
#include "arm-cla-regs.h"
@@ -21,10 +23,12 @@
*
* Immutable state:
* @cpu: The CPU this CLA is attached to.
+ * @regs: Registers accessed by the kernel.
* @dev: The platform device.
*/
struct cla_dev {
unsigned int cpu;
+ void __iomem *regs;
struct device *dev;
};
@@ -35,4 +39,21 @@ struct cla_dev {
#define cla_err(dev, fmt, ...) \
dev_err((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
+static inline u64 cla_reg_read(struct cla_dev *dev, off_t reg)
+{
+ return readq_relaxed(dev->regs + reg);
+}
+
+static inline void cla_reg_write(struct cla_dev *dev, off_t reg, u64 val)
+{
+ return writeq_relaxed(val, dev->regs + reg);
+}
+
+int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp);
+int cla_op_reset(struct cla_dev *dev, unsigned int accid);
+int cla_op_regread(struct cla_dev *dev, unsigned int accid, unsigned int regidx,
+ size_t nregs, u64 *regs);
+int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
+ unsigned int regidx, size_t nregs, u64 *regs);
+
#endif /* _ARM_CLA_H_ */
diff --git a/drivers/misc/arm-cla/cla-init.c b/drivers/misc/arm-cla/cla-init.c
new file mode 100644
index 000000000000..3d1f47592842
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-init.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - probing and initialization
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/module.h>
+
+MODULE_DESCRIPTION("Arm Core Local Accelerator");
+MODULE_AUTHOR("Arm Limited");
+MODULE_LICENSE("GPL");
diff --git a/drivers/misc/arm-cla/cla-ops.c b/drivers/misc/arm-cla/cla-ops.c
new file mode 100644
index 000000000000..d594344a2eb4
--- /dev/null
+++ b/drivers/misc/arm-cla/cla-ops.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Arm CLA driver - Launch operations
+ *
+ * Copyright 2026 Arm Limited.
+ */
+
+#include <linux/iopoll.h>
+
+#include "arm-cla.h"
+
+/* Time to wait between two LRESP reads */
+#define CLA_LRESP_DELAY_US 1
+
+/*
+ * Time to wait for LRESP_PENDING to clear. Commands like REGREAD should
+ * complete in a few cycles, but ENTERSR and RESET may need to clean up
+ * some very large states depending on the work interrupted, and may need
+ * 1us or more.
+ */
+#define CLA_LRESP_TIMEOUT_US 100
+
+enum cla_launch_data_mode {
+ CLA_DATA_NONE,
+ CLA_DATA_IN,
+ CLA_DATA_OUT,
+};
+
+struct cla_launch {
+ u8 op; /* opcode */
+ u8 ndata_m1; /* Data size minus 1 */
+ u8 accid; /* Accelerator ID */
+ bool seq; /* part of compound cmd */
+ u32 regidx; /* Register index */
+
+ enum cla_launch_data_mode data_mode;
+ u64 *data; /* In/out data */
+
+ u8 errcode; /* Output error code */
+};
+
+/**
+ * cla_op_launch - Launch operation.
+ * @dev: CLA device.
+ * @launch: LAUNCH settings.
+ *
+ * 1. If data_mode is %CLA_DATA_IN, write DATA registers.
+ * 2. Launch the operation, and wait for the response.
+ * 3. If data_mode is %CLA_DATA_OUT, read DATA registers into @launch->data.
+ *
+ * Return:
+ * * %0 - Success.
+ * * %-ETIMEDOUT - LAUNCH timed out (possibly no CLA at this address).
+ * Unless LRESP_PENDING eventually clears, this is
+ * unrecoverable.
+ * * %-ENODEV - Accelerator not available.
+ * * %-EBUSY - Accelerator is busy.
+ * * %-EIO - LAUNCH error. @launch->errcode contains the LRESP
+ * error code.
+ */
+static int cla_op_launch(struct cla_dev *dev, struct cla_launch *launch)
+{
+ int i;
+ int ret;
+ u64 lresp;
+
+ if (WARN_ON(smp_processor_id() != dev->cpu))
+ return -EINVAL;
+
+ if (launch->data_mode == CLA_DATA_IN)
+ for (i = 0; i < launch->ndata_m1 + 1; i++)
+ cla_reg_write(dev, CLA_REG_DATA(i), launch->data[i]);
+
+ /*
+ * No barrier needed because accesses use Device-nGnRE, within the same
+ * memory-mapped peripheral, so accesses arrive at the endpoint in
+ * program order.
+ */
+ cla_reg_write(dev, CLA_REG_LAUNCH,
+ FIELD_PREP(CLA_LAUNCH_OP, launch->op) |
+ FIELD_PREP(CLA_LAUNCH_NDATA_M1, launch->ndata_m1) |
+ FIELD_PREP(CLA_LAUNCH_ACCID, launch->accid) |
+ FIELD_PREP(CLA_LAUNCH_SEQ, launch->seq) |
+ FIELD_PREP(CLA_LAUNCH_REGIDX, launch->regidx));
+
+ ret = cla_op_wait_lresp(dev, &lresp);
+ if (ret) {
+ cla_err(dev, "launch failed with %d\n", ret);
+ return ret;
+ }
+
+ switch (FIELD_GET(CLA_LRESP_CODE, lresp)) {
+ case CLA_LRESP_OK:
+ break;
+ case CLA_LRESP_UNAVAIL:
+ return -ENODEV;
+ case CLA_LRESP_BUSY:
+ return -EBUSY;
+ case CLA_LRESP_ERROR:
+ launch->errcode = FIELD_GET(CLA_LRESP_ERRCODE, lresp);
+ return -EIO;
+ }
+
+ if (launch->data_mode == CLA_DATA_OUT)
+ for (i = 0; i < launch->ndata_m1 + 1; i++)
+ launch->data[i] = cla_reg_read(dev, CLA_REG_DATA(i));
+
+ return 0;
+}
+
+/**
+ * cla_op_wait_lresp - Wait for any LAUNCH op to complete.
+ * @dev: CLA device.
+ * @lresp: last LRESP value read.
+ *
+ * Return: 0 on success, -ETIMEDOUT in case of timeout.
+ */
+int cla_op_wait_lresp(struct cla_dev *dev, u64 *lresp)
+{
+ return readq_relaxed_poll_timeout_atomic(dev->regs + CLA_REG_LRESP,
+ *lresp, FIELD_GET(CLA_LRESP_PENDING, *lresp) == 0,
+ CLA_LRESP_DELAY_US, CLA_LRESP_TIMEOUT_US);
+}
+
+/**
+ * cla_op_reset - Launch RESET operation for this accelerator.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ *
+ * Return: 0 on success, 1 if there is no accelerator with this ID, or an error.
+ */
+int cla_op_reset(struct cla_dev *dev, unsigned int accid)
+{
+ int ret;
+ struct cla_launch launch = {
+ .op = CLA_LAUNCH_OP_RESET,
+ .accid = accid,
+ };
+
+ ret = cla_op_launch(dev, &launch);
+ if (ret == -EIO && launch.errcode == CLA_ERRCODE_NOACC)
+ return 1;
+ return ret;
+}
+
+static int cla_op_access_reg(struct cla_dev *dev, u8 op,
+ enum cla_launch_data_mode data_mode,
+ unsigned int accid, unsigned int regidx,
+ size_t nregs, u64 *regs)
+{
+ int ret = -EINVAL;
+ unsigned long max_regidx;
+ struct cla_launch launch = {
+ .op = op,
+ .accid = accid,
+ .data_mode = data_mode,
+ };
+
+ switch (op) {
+ case CLA_LAUNCH_OP_REGREAD:
+ case CLA_LAUNCH_OP_REGWRITE:
+ max_regidx = 0x100000000;
+ break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ if (WARN_ON(regidx + nregs > max_regidx))
+ return -EINVAL;
+
+ /* 1 to 8 registers accessed at a time, within the same 8-reg group */
+ while (nregs > 0) {
+ unsigned int reg_group = ALIGN_DOWN(regidx, 8);
+ unsigned int max_reg = min(regidx + nregs, reg_group + 8);
+ unsigned int ndata = max_reg - regidx;
+
+ launch.ndata_m1 = ndata - 1;
+ launch.regidx = regidx;
+ launch.data = regs;
+
+ ret = cla_op_launch(dev, &launch);
+ if (ret)
+ break;
+
+ regidx += ndata;
+ regs += ndata;
+ nregs -= ndata;
+ }
+
+ return ret;
+}
+
+/**
+ * cla_op_regread - Launch REGREAD operations.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @regidx: first register index.
+ * @nregs: number of registers. Can be greater than 8 (accessed with multiple
+ * REGREAD operations).
+ * @regs: array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_regread(struct cla_dev *dev, unsigned int accid,
+ unsigned int regidx, size_t nregs, u64 *regs)
+{
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_REGREAD, CLA_DATA_OUT,
+ accid, regidx, nregs, regs);
+}
+
+/**
+ * cla_op_regwrite - Launch REGWRITE operations.
+ * @dev: CLA device.
+ * @accid: accelerator ID.
+ * @regidx: first register index.
+ * @nregs: number of registers. Can be greater than 8 (accessed with multiple
+ * REGWRITE operations).
+ * @regs: array of length @nregs.
+ *
+ * Return: 0 on success, or an error.
+ */
+int cla_op_regwrite(struct cla_dev *dev, unsigned int accid,
+ unsigned int regidx, size_t nregs, u64 *regs)
+{
+ return cla_op_access_reg(dev, CLA_LAUNCH_OP_REGWRITE, CLA_DATA_IN,
+ accid, regidx, nregs, regs);
+}
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 1/8] misc/arm-cla: Add driver skeleton and documentation
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
In-Reply-To: <20260717104759.123203-1-ryan.roberts@arm.com>
From: Jean-Philippe Brucker <jpb@kernel.org>
Add the initial Kconfig and build-system plumbing for the Arm Core Local
Accelerator driver.
Introduce the common driver header and register definitions used by
later CLA support. The definitions cover the CLA MMIO frame, launch
response and status fields, standard accelerator registers, launch
opcodes, error codes and memory translation context state.
Add documentation describing the CLA programming model, its CPU-local
MMIO access rules, userspace assignment model, domain grouping and
expected boot state.
Co-developed-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Jean-Philippe Brucker <jpb@kernel.org>
---
Documentation/misc-devices/arm-cla.rst | 206 +++++++++++++++++++++++++
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/arm-cla/Kconfig | 10 ++
drivers/misc/arm-cla/Makefile | 1 +
drivers/misc/arm-cla/arm-cla-regs.h | 177 +++++++++++++++++++++
drivers/misc/arm-cla/arm-cla.h | 38 +++++
7 files changed, 434 insertions(+)
create mode 100644 Documentation/misc-devices/arm-cla.rst
create mode 100644 drivers/misc/arm-cla/Kconfig
create mode 100644 drivers/misc/arm-cla/Makefile
create mode 100644 drivers/misc/arm-cla/arm-cla-regs.h
create mode 100644 drivers/misc/arm-cla/arm-cla.h
diff --git a/Documentation/misc-devices/arm-cla.rst b/Documentation/misc-devices/arm-cla.rst
new file mode 100644
index 000000000000..dfe2ab4a10a0
--- /dev/null
+++ b/Documentation/misc-devices/arm-cla.rst
@@ -0,0 +1,206 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================================
+Arm Core Local Accelerator (CLA)
+================================
+
+Arm CLA is an interface local to a CPU for programming accelerators that access
+memory via the CPU's MMU:
+
+ ┌───────┬───────┬───────┐ ┌───────┐
+ │ CPU │ MMU │ CLA │ │ Accel │─┐
+ │ │ │ │ │ │ │
+ │ ---MMIO--> <--> │ │
+ │ │ │ │ │ │ │
+ │ │ ,-----DMA-----> │ │
+ └───────┴────|──┴───────┘ └───────┘ │
+ ┌──────────v────────┐ └───────┘
+ │ Caches and memory │
+ └───────────────────┘
+
+
+Hardware
+========
+
+The CLA supports up to 8 attached accelerators, which are accessed by
+programming the CLA's MMIO registers. Operations are launched to an accelerator
+and are polled for completion. CLA does not raise interrupts.
+
+ CPU CLA Accel
+ |--- write DATA[7:0] -->| |
+ |--- write LAUNCH ----->|---- launch ---->|
+ |<--- poll LRESP -------| |
+ | | |
+ |<--- poll STATUS ------|<--- complete ---|
+
+Each operation can take a 512-bit payload in the DATA registers. After handling
+a LAUNCH write, CLA indicates the launch status in the LRESP register. A further
+operation can only be launched after LRESP indicates completion of the previous
+launch.
+
+Some operations continue to run asynchronously on the accelerator after launch
+completion. In this case progress is tracked by polling the STATUS register.
+When the CLA updates the STATUS register, it also raises an event which will
+wake an in-progress WFE (wait for event) instruction on the local CPU.
+
+The CLA's MMIO registers are not accessible from remote CPUs. Although each CLA
+has a unique physical address, accesses from remote CPUs are read as zero and
+write ignored.
+
+The CLA registers are accessible from four different Privilege Level (PL)
+frames, with usage inteded to map to EL0 - EL3. The PLxCTRL registers may be
+written via a higher PL frame to suppress access to accelerators via a lower PL
+frame.
+
+The CPU and CLA share an MMU, although FEAT_TTCNP (common not private) is
+implemented, allowing both CPU and CLA to independently opt into and out of
+sharing TLB entries at runtime. TLB invalidation is performed via the CPU TLBI
+instructions; any TLBI instruction that targets the CLA's local CPU will also
+implicitly target the CLA.
+
+CLA has its own set of Memory Translation Context (MTC) registers, distinct from
+the CPU. A PL can set the MTC registers corresponding to an equivalent CPU
+Exception Level (EL) (eg. TCR_EL2 configurable only from the PL2 and PL3 MMIO
+frame).
+
+Faults during address translation are reported by the accelerator in its
+registers and in STATUS. While polling for work completion, software fixes up
+the faults and notifies the accelerator with RESOLVE operations.
+
+Accesses to the MMIO registers must be aligned 64-bit loads and stores, and the
+registers are mapped with Device-nGnRE attribute. Invalid accesses (unaligned,
+atomic, badly sized, etc) to the MMIO frame are either read as zero and write
+ignored, or cause a data abort, depending on the platform. Invalid access will
+never cause an SError.
+
+
+Inter-Accelerator Communication
+-------------------------------
+
+On some platforms, multiple accelerators, each attached to a separate CLA within
+a cluster, are also directly connected to each other via a shared bus to
+accelerate cooperation between accelerators. The accelerators sharing a bus
+cannot be isolated from each other. When collaborative operations are launched
+on each of the participating accelerators, they synchronize over the bus,
+stalling until all are ready.
+
+
+Intended SW Usage Model
+=======================
+
+CLA is designed for its PL0 MMIO frame to be mapped into user space and for user
+space to directly launch accelerator operations and poll for completion. It has
+been observed that for some use cases, the operation execution time is small and
+a trip through the kernel would consume a significant amount of the CPU budget
+for preparing the next operation leading to a significant reduction in bandwidth
+through the accelerator.
+
+User space software is expected to create a thread to drive each CLA it is
+using, and for each thread to be pinned to the CLA's local CPU.
+
+Software should rely on WFE (wait for event) to reduce power consumption when
+polling STATUS.
+
+The CLA is intended to be configured, by privileged software via PL1 and/or PL2,
+so that it shares virtual addresses with the process to which it is assigned
+(SVA). In practice this means configuring the CLA's MTC to point to the same
+page tables and use the same ASID (and VMID if relevant) as the process to which
+it is assigned. This ensures the architectural TLB invalidations also correctly
+target the CLA's TLB entries.
+
+We investigated the possibility of having the CLA driver allocate private page
+tables, private ASIDs/VMIDs and implement an MMU notifier for invalidation, but
+that suffers from 2 issues; there is a possibility of over-invalidation since
+the ASID and VMID spaces overlap with the CPU's (minor), and when issuing a
+TLBI, VMID is implicitly taken from CPU's VTTBR_EL2.VMID, which won't match the
+CLA's private VMID - therefore, for a virtualization scenario, TLB invalidation
+becomes impossible (major).
+
+Because the CLA has its own MTC registers, it is correct and safe for it to be
+executing an operation on behalf of user process A, while its local CPU is
+executing a thread from user process B.
+
+
+Difficulties for software to deal with
+--------------------------------------
+
+Although each CLA is attached to a single CPU, not all CPUs have a CLA, and CLAs
+may have a different set of accelerators attached. Users need to probe around to
+find a suitable accelerator and bind their process to it.
+
+Since remote CPUs can't access the CLA, dealing with CPU hotplug migrating tasks
+is challenging. And virtualization breaks down if the hypervisor cannot
+guarantee that a vCPU is pinned to a CPU.
+
+Saving and restoring the internal state of the accelerator is an optional
+feature. Current platforms only support it when the accelerator is idle, so
+preempting an accelerator causes work cancellation. Software must carefully
+consider how to balance forward-progress guarantees with preemption latency.
+
+
+Driver
+======
+
+Booting
+-------
+
+A host expects to be booted with CLAs in the following state:
+- All attached accelerators have STATUS_IDLE set.
+- PL2CTRL: AVAIL all enabled (no accelerator request made at runtime).
+ DBG at the firmware's discretion, preferably all enabled.
+- TSCTRLOWNER.PL, TSOFFOWNER.PL, PMUOWNER.PL all EL2.
+
+A guest expects to be booted with CLAs in the following state:
+- All attached accelerators have STATUS_IDLE set.
+- PL1CTRL: AVAIL all enabled (no accelerator request made at runtime).
+ DBG at the hypervisor's discretion, preferably all enabled.
+
+Linux discovers the base address of each CLA device in the firmware tables and
+creates platform devices. cla_dev behaves mostly like a regular platform device,
+but it can only be accessed from one specific CPU. CPU hotplug notifiers probe
+and teardown the CLA device.
+
+CLAs may be grouped into domains. The topology is described in the firmware
+tables, and the driver creates cla_domain objects containing one or more
+cla_dev.
+
+
+Assignment to userspace
+-----------------------
+
+A char device provides enumeration and mmap abilities to userspace. The user
+task queries the driver to find a suitable set of CLAs, and mmaps their
+registers. For each mapped domain, the driver creates a cla_ctx context. When
+the user accesses the registers, the driver's fault handler queues the cla_ctx
+and waits for assignment. A context reassignment switches the whole cla_domain,
+by calling each CPU in the domain to switch the CLA context. A time slice is
+given to each context before being deassigned.
+
+In order to access the registers, the task must be bound to the CLA's CPU with
+sched_setaffinity(). Accesses from remote CPUs are ignored. If the CPU gets
+offlined, the task is migrated to an online CPU, and the driver disables the
+offlined CLA. User space may choose to use a mechanism such as restartable
+sequences to be notified when its task has been migrated away from its intended
+CLA.
+
+The accelerators access memory using the CPU's MMU. When assigning a context,
+the driver sets up the CLA's Memory Translation Context (MTC) with the page
+directory (TTBR), address space ID (ASID) and configuration (TCR) of the
+context's mm. When receiving translation faults from the accelerator, the user
+space task accesses the faulting address from the CPU to trigger the fault to be
+fixed up by the kernel before launching a RESOLVE operation to the accelerator.
+
+
+Power management
+----------------
+
+When a CPU enters a deep low-power state, then depending on the platform, the
+attached CLA and accelerators might not retain their state. In that case the
+firmware is expected to save the CLA and accelerator states before entering CPU
+low-power state, and restore them after exiting. The CLA driver does not expect
+to notice a CPU deep idle. However some accelerators do not support context
+saving, in which case userspace will notice from the STATUS register that the
+work was canceled during deep idle. Given that userspace would be polling
+STATUS, the CPU is unlikely to enter deep idle while the CLA is running. To
+ensure forward-progress the admin can disable deep idle states (see
+Documentation/admin-guide/pm/cpuidle.rst).
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index 390256ed91f4..a2491b5da224 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -651,4 +651,5 @@ source "drivers/misc/mchp_pci1xxxx/Kconfig"
source "drivers/misc/keba/Kconfig"
source "drivers/misc/amd-sbi/Kconfig"
source "drivers/misc/rp1/Kconfig"
+source "drivers/misc/arm-cla/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index fed47c7672b9..51d878822c1f 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -74,3 +74,4 @@ obj-$(CONFIG_MCHP_LAN966X_PCI) += lan966x-pci.o
obj-y += keba/
obj-y += amd-sbi/
obj-$(CONFIG_MISC_RP1) += rp1/
+obj-$(CONFIG_ARM_CLA) += arm-cla/
diff --git a/drivers/misc/arm-cla/Kconfig b/drivers/misc/arm-cla/Kconfig
new file mode 100644
index 000000000000..f8436a12c1e6
--- /dev/null
+++ b/drivers/misc/arm-cla/Kconfig
@@ -0,0 +1,10 @@
+config ARM_CLA
+ tristate "Arm Core Local Accelerator"
+ default n
+ depends on ARM64
+ help
+ Arm Core Local Accelerator (CLA) is coupled with a specific CPU and
+ provides an interface to enumerate and access attached hardware
+ accelerators. It is only accessible from its associated CPU.
+
+ If unsure, say N.
diff --git a/drivers/misc/arm-cla/Makefile b/drivers/misc/arm-cla/Makefile
new file mode 100644
index 000000000000..a4e40e534e6a
--- /dev/null
+++ b/drivers/misc/arm-cla/Makefile
@@ -0,0 +1 @@
+# SPDX-License-Identifier: GPL-2.0-only
diff --git a/drivers/misc/arm-cla/arm-cla-regs.h b/drivers/misc/arm-cla/arm-cla-regs.h
new file mode 100644
index 000000000000..fcd187ecb777
--- /dev/null
+++ b/drivers/misc/arm-cla/arm-cla-regs.h
@@ -0,0 +1,177 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Arm CLA driver - register definitions
+ *
+ * Copyright 2026 Arm Limited.
+ */
+#ifndef _ARM_CLA_REGS_H_
+#define _ARM_CLA_REGS_H_
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+
+/* Registers */
+#define CLA_REG_DATA(i) (0x00 + (8 * (i)))
+#define CLA_REG_LAUNCH 0x40
+#define CLA_REG_LRESP 0x48
+#define CLA_REG_PL0CTRL 0x60
+#define CLA_REG_PL1CTRL 0x68
+#define CLA_REG_PL2CTRL 0x70
+#define CLA_REG_EVENT 0x78
+#define CLA_REG_STATUS(i) (0x80 + (8 * (i)))
+#define CLA_REG_CLAAIDR 0xC0
+
+#define CLA_LAUNCH_OP GENMASK(3, 0)
+#define CLA_LAUNCH_NDATA_M1 GENMASK(6, 4)
+#define CLA_LAUNCH_SEQ BIT(7)
+#define CLA_LAUNCH_ACCID GENMASK(10, 8)
+#define CLA_LAUNCH_REGIDX GENMASK(63, 32)
+
+#define CLA_LRESP_PENDING BIT(0)
+#define CLA_LRESP_CODE GENMASK(2, 1)
+#define CLA_LRESP_ERRCODE GENMASK(7, 3)
+#define CLA_LRESP_DATANZ BIT(15)
+
+#define CLA_PLxCTRL_AVAIL BIT(0)
+#define CLA_PLxCTRL_DBGPERM GENMASK(3, 1)
+#define CLA_PLxCTRL_PREP(accid, v) ((u64)(v) << (8 * (accid)))
+
+#define CLA_STATUS_AVAIL BIT(0)
+#define CLA_STATUS_DMB BIT(1)
+#define CLA_STATUS_EABORT BIT(2)
+#define CLA_STATUS_IDLE BIT(4)
+#define CLA_STATUS_READY BIT(5)
+#define CLA_STATUS_FAULT BIT(6)
+#define CLA_STATUS_EXCEPT BIT(7)
+#define CLA_STATUS_SRMODE BIT(8)
+#define CLA_STATUS_USER GENMASK(63, 16)
+
+/* Some useful values for sanity checks */
+#define CLA_STATUS_STATE_IDLE (CLA_STATUS_AVAIL | \
+ CLA_STATUS_IDLE | \
+ CLA_STATUS_READY)
+#define CLA_STATUS_STATE_SRMODE (CLA_STATUS_AVAIL | \
+ CLA_STATUS_IDLE | \
+ CLA_STATUS_READY | \
+ CLA_STATUS_SRMODE)
+/* Bits we care about when checking the state */
+#define CLA_STATUS_STATE_MASK (CLA_STATUS_AVAIL | \
+ CLA_STATUS_EABORT | \
+ CLA_STATUS_IDLE | \
+ CLA_STATUS_READY | \
+ CLA_STATUS_FAULT | \
+ CLA_STATUS_EXCEPT | \
+ CLA_STATUS_SRMODE)
+
+/* Standard accelerator registers */
+#define CLA_REG_IIDR 0x0000
+#define CLA_REG_DEVARCH 0x0001
+#define CLA_REG_REVIDR 0x0002
+#define CLA_REG_IASSIZE 0x0003
+#define CLA_REG_ACAP 0x0004
+#define CLA_REG_FSARV 0x001f
+#define CLA_REG_FSAR(n) (0x0020 + (n))
+#define CLA_REG_TSCTRLOWNER 0x00c0
+#define CLA_REG_TSCTRL 0x00c8
+#define CLA_REG_TSOFFOWNER 0x00d0
+#define CLA_REG_TSVOFF 0x00d8
+#define CLA_REG_TSPOFF 0x00d9
+#define CLA_REG_PMUOWNER 0x0100
+#define CLA_REG_PMURESET 0x0108
+#define CLA_REG_PMUCTRL 0x0110
+#define CLA_REG_PMUSNAP 0x0111
+#define CLA_REG_PMUEVT(n) (0x0120 + (n))
+#define CLA_REG_PMUCNT(n) (0x0140 + (n))
+#define CLA_REG_PMUSCNT(n) (0x0160 + (n))
+#define CLA_REG_IASn 0x8000
+
+#define CLA_IIDR_PRODUCTID GENMASK(31, 20)
+#define CLA_IIDR_VARIANT GENMASK(19, 16)
+#define CLA_IIDR_REVISION GENMASK(15, 12)
+#define CLA_IIDR_IMPLEMENTER GENMASK(11, 0)
+
+#define CLA_DEVARCH_ARCHITECT GENMASK(31, 21)
+#define CLA_DEVARCH_PRESENT BIT(20)
+#define CLA_DEVARCH_REVISION GENMASK(19, 16)
+#define CLA_DEVARCH_ARCHID GENMASK(15, 0)
+
+#define CLA_REVIDR_REVISION GENMASK(31, 0)
+
+#define CLA_ACAP_SROP BIT(0)
+#define CLA_ACAP_REGSTATE BIT(1)
+#define CLA_ACAP_PMUCNTS GENMASK(4, 2)
+#define CLA_ACAP_TS BIT(5)
+
+#define CLA_FSAR_READ BIT(0)
+#define CLA_FSAR_WRITE BIT(1)
+#define CLA_FSAR_ADDR GENMASK(63, 6)
+
+#define CLA_TSCTRLOWNER_PL GENMASK(1, 0)
+#define CLA_TSCTRL_TS GENMASK(1, 0)
+#define CLA_TSOFFOWNER_PL GENMASK(1, 0)
+
+#define CLA_TSCTRL_ZERO 0
+#define CLA_TSCTRL_VIRTUAL 1
+#define CLA_TSCTRL_GUESTPHYSICAL 2
+#define CLA_TSCTRL_PHYSICAL 3
+
+#define CLA_PMUCTRL_EN BIT(0)
+#define CLA_PMUOWNER_PL GENMASK(1, 0)
+
+/* LAUNCH operations */
+#define CLA_LAUNCH_OP_RESET 0
+#define CLA_LAUNCH_OP_CMD 1
+#define CLA_LAUNCH_OP_CMDNR 2
+#define CLA_LAUNCH_OP_ENTERSR 4
+#define CLA_LAUNCH_OP_EXITSR 5
+#define CLA_LAUNCH_OP_SAVE 6
+#define CLA_LAUNCH_OP_RESTORE 7
+#define CLA_LAUNCH_OP_RESOLVE 9
+#define CLA_LAUNCH_OP_REGREAD 10
+#define CLA_LAUNCH_OP_REGWRITE 11
+#define CLA_LAUNCH_OP_SETCTX 12
+#define CLA_LAUNCH_OP_GETCTX 13
+
+/* Return codes */
+#define CLA_LRESP_OK 0
+#define CLA_LRESP_UNAVAIL 1
+#define CLA_LRESP_BUSY 2
+#define CLA_LRESP_ERROR 3
+
+#define CLA_ERRCODE_CSINT 0
+#define CLA_ERRCODE_CSOF 1
+#define CLA_ERRCODE_NOTIDLE 2
+#define CLA_ERRCODE_PERM 3
+#define CLA_ERRCODE_NOACC 4
+#define CLA_ERRCODE_INVAL 5
+#define CLA_ERRCODE_RESET 6
+
+/* Memory translation context */
+#define CLA_MTC_REGIDX_PL1 0
+#define CLA_MTC_REGIDX_PL2 64
+#define CLA_MTC_PL_SIZE 64
+
+/* Common register offsets */
+#define CLA_MTC_PSTATE 0
+#define CLA_MTC_TTBR0 1
+#define CLA_MTC_TTBR1 2
+#define CLA_MTC_TCR 3
+#define CLA_MTC_SCTLR 4
+#define CLA_MTC_MAIR 5
+#define CLA_MTC_TCR2 8
+
+/* EL2 specific register offsets */
+#define CLA_MTC_HCR_EL2 80
+#define CLA_MTC_VTTBR_EL2 81
+#define CLA_MTC_VTCR_EL2 82
+
+#define CLA_MTC_PSTATE_EL GENMASK(1, 0)
+#define CLA_MTC_PSTATE_PAN BIT(2)
+
+#define CLA_SRSTATE_0_SROP BIT(0)
+#define CLA_SRSTATE_0_REGSTATE GENMASK(15, 1)
+#define CLA_SRSTATE_1_STATUS GENMASK(63, 0)
+#define CLA_SRSTATE_2_SRACTIVE GENMASK(1, 0)
+#define CLA_SRSTATE_2_ADDR_MASK GENMASK(63, 6)
+
+#endif /* _ARM_CLA_REGS_H_ */
diff --git a/drivers/misc/arm-cla/arm-cla.h b/drivers/misc/arm-cla/arm-cla.h
new file mode 100644
index 000000000000..f265d7b60268
--- /dev/null
+++ b/drivers/misc/arm-cla/arm-cla.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Arm CLA driver - internal definitions
+ *
+ * Copyright 2026 Arm Limited.
+ */
+#ifndef _ARM_CLA_H_
+#define _ARM_CLA_H_
+
+#include <linux/types.h>
+
+#include "arm-cla-regs.h"
+
+/* Number of accelerators per CLA */
+#define CLA_NUM_ACC 8
+#define CLA_NUM_DATA_REGS 8
+#define CLA_SRSTATE_LEN 8
+
+/**
+ * struct cla_dev - CLA device
+ *
+ * Immutable state:
+ * @cpu: The CPU this CLA is attached to.
+ * @dev: The platform device.
+ */
+struct cla_dev {
+ unsigned int cpu;
+ struct device *dev;
+};
+
+#define cla_dbg(dev, fmt, ...) \
+ dev_dbg((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
+#define cla_info(dev, fmt, ...) \
+ dev_info((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
+#define cla_err(dev, fmt, ...) \
+ dev_err((dev)->dev, "[%u] " fmt, (dev)->cpu, ##__VA_ARGS__)
+
+#endif /* _ARM_CLA_H_ */
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v1 0/8] Arm Core Local Accelerator Driver
From: Ryan Roberts @ 2026-07-17 10:47 UTC (permalink / raw)
To: Greg Kroah-Hartman, Arnd Bergmann, Catalin Marinas, Will Deacon,
Mark Rutland, Jean-Philippe Brucker, Oded Gabbay, Jonathan Corbet
Cc: Ryan Roberts, linux-kernel, linux-arm-kernel, dri-devel,
linux-doc
Hi All,
This RFC introduces a driver for the Arm Core Local Accelerator (CLA), a
CPU-local interface for programming attached accelerators. While the interface
is agnostic to the accelerator type, the initial (and currently only) target is
a compute engine.
The RFC aims to engage the community and get feedback on some key aspects. This
will help plan our approach for eventual upstreaming. The patches implement a
bare bones driver to aid discussion. Arm plans to publish the CLA spec in
future, but for now I hope the documentation included with patch 1 suffices.
Note that the CLA is not part of the Arm Architecture.
Patch 1 documents the hardware and driver design, and adds a driver skeleton.
Patches 2-4 initialize and probe the device. Patches 5-8 add context management
for switching the device between different users.
Aspects I'm seeking feedback for:
* The general structure of the driver: The current shape addresses the
performance requirements we have for the use cases, and is therefore our
preferred approach. But there are some unusual aspects due to the HW design.
* Driver interaction with architectural support: I've opted to treat the CLA as
a device rather than a CPU extension, so it's implemented as a (mostly)
self-contained driver. It has some unavoidable coupling to the arch code since
it needs to share page tables and ASIDs (arm64_mm_context_[put|get]()).
* The location of the driver: although most accelerators will likely be compute
ones, CLA is a generic MMIO interface that could handle any kind of
accelerator. It's currently implemented as a misc driver. accel is another
potential option, but given the current SVA approach, we would not use any of
the services (memory-management or otherwise) that accel provides.
* User space availability: The kernel driver exposes the capabilities of the
hardware to user space. Arm plans to open source a user space driver, but does
not yet have any committed date. I'd like to understand if the availability of
this component will be a prerequisite for upstream acceptance of the kernel
driver; either way, I'm hoping we can at least progress with some discussion
in its absence.
I'm deliberately constraining the scope to bare-metal support for now.
Virtualization is something we are considering (and have prototyped), but plan
to post a separate RFC for that as follow-up, once we have agreement on
direction for the bare-metal driver.
Source, along with some tests, is available at [1]. Patches based on v7.2-rc3.
[1] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/cla-driver-v1-rfc-tests
Thanks,
Ryan
Jean-Philippe Brucker (5):
misc/arm-cla: Add driver skeleton and documentation
misc/arm-cla: Add launch operation helpers
misc/arm-cla: Probe firmware-described devices
misc/arm-cla: Initialize devices on CPU bringup
misc/arm-cla: Accelerator context save and restore
Ryan Roberts (3):
misc/arm-cla: Set up memory translation context
misc/arm-cla: Manage domain contexts
misc/arm-cla: Add userspace interface
Documentation/misc-devices/arm-cla.rst | 206 ++++++++++
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/arm-cla/Kconfig | 10 +
drivers/misc/arm-cla/Makefile | 13 +
drivers/misc/arm-cla/arm-cla-regs.h | 179 +++++++++
drivers/misc/arm-cla/arm-cla.h | 296 +++++++++++++++
drivers/misc/arm-cla/cla-ctx.c | 142 +++++++
drivers/misc/arm-cla/cla-init.c | 503 +++++++++++++++++++++++++
drivers/misc/arm-cla/cla-mtc.c | 139 +++++++
drivers/misc/arm-cla/cla-ops.c | 342 +++++++++++++++++
drivers/misc/arm-cla/cla-regs.c | 263 +++++++++++++
drivers/misc/arm-cla/cla-sched.c | 474 +++++++++++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 187 +++++++++
drivers/misc/arm-cla/cla-user.c | 351 +++++++++++++++++
include/uapi/linux/arm-cla.h | 207 ++++++++++
16 files changed, 3314 insertions(+)
create mode 100644 Documentation/misc-devices/arm-cla.rst
create mode 100644 drivers/misc/arm-cla/Kconfig
create mode 100644 drivers/misc/arm-cla/Makefile
create mode 100644 drivers/misc/arm-cla/arm-cla-regs.h
create mode 100644 drivers/misc/arm-cla/arm-cla.h
create mode 100644 drivers/misc/arm-cla/cla-ctx.c
create mode 100644 drivers/misc/arm-cla/cla-init.c
create mode 100644 drivers/misc/arm-cla/cla-mtc.c
create mode 100644 drivers/misc/arm-cla/cla-ops.c
create mode 100644 drivers/misc/arm-cla/cla-regs.c
create mode 100644 drivers/misc/arm-cla/cla-sched.c
create mode 100644 drivers/misc/arm-cla/cla-topology.c
create mode 100644 drivers/misc/arm-cla/cla-user.c
create mode 100644 include/uapi/linux/arm-cla.h
--
2.43.0
^ permalink raw reply
* Re: [PATCH v1] pnp: Documentation improvements
From: Rafael J. Wysocki (Intel) @ 2026-07-17 10:19 UTC (permalink / raw)
To: Uwe Kleine-König (The Capable Hub)
Cc: Rafael J. Wysocki, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel
In-Reply-To: <aloAhMQE2mWwwzpY@monoceros>
On Fri, Jul 17, 2026 at 12:16 PM Uwe Kleine-König (The Capable Hub)
<u.kleine-koenig@baylibre.com> wrote:
>
> Hello,
>
> On Tue, Jun 09, 2026 at 04:51:17PM +0200, Uwe Kleine-König (The Capable Hub) wrote:
> > - Consistently use named initializers and simplify sentinel
> > - Skip assignment to .driver_data if all are 0
> > - Use consistent spacing to match Linux coding style
> > - Fix prototype of probe function
> > - s/pnp_id/pnp_device_id/
> > - Drop non-existing .card_id_table
> >
> > Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
>
> Who is supposed to pick up this patch? Is this still in someone's queue
> that I can expect to get some feedback for it? Silent application would
> also be fine to me.
>
> Having said that, the patch still applies fine to next-20260716.
I'll pick it up, but please send it to linux-acpi@vger.kernel.org for
easier processing.
Thanks!
^ permalink raw reply
* Re: [PATCH v1] pnp: Documentation improvements
From: Uwe Kleine-König (The Capable Hub) @ 2026-07-17 10:16 UTC (permalink / raw)
To: Rafael J. Wysocki; +Cc: Jonathan Corbet, Shuah Khan, linux-doc, linux-kernel
In-Reply-To: <20260609145117.1355753-2-u.kleine-koenig@baylibre.com>
[-- Attachment #1: Type: text/plain, Size: 720 bytes --]
Hello,
On Tue, Jun 09, 2026 at 04:51:17PM +0200, Uwe Kleine-König (The Capable Hub) wrote:
> - Consistently use named initializers and simplify sentinel
> - Skip assignment to .driver_data if all are 0
> - Use consistent spacing to match Linux coding style
> - Fix prototype of probe function
> - s/pnp_id/pnp_device_id/
> - Drop non-existing .card_id_table
>
> Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
Who is supposed to pick up this patch? Is this still in someone's queue
that I can expect to get some feedback for it? Silent application would
also be fine to me.
Having said that, the patch still applies fine to next-20260716.
Best regards
Uwe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* Re: [PATCH v3 6/6] Documentation/kernel-parameters: add/update printk_delay/boot_delay
From: Petr Mladek @ 2026-07-17 9:10 UTC (permalink / raw)
To: Steven Rostedt
Cc: Andrew Murray, John Ogness, Sergey Senozhatsky, Jonathan Corbet,
Shuah Khan, Russell King, Florian Fainelli,
Broadcom internal kernel review list, Ray Jui, Scott Branden,
Andrew Morton, Greg Kroah-Hartman, Sebastian Andrzej Siewior,
Clark Williams, linux-kernel, linux-doc, linux-arm-kernel,
linux-rpi-kernel, linux-rt-devel
In-Reply-To: <20260714175131.1edfc880@gandalf.local.home>
On Tue 2026-07-14 17:51:31, Steven Rostedt wrote:
> On Sun, 12 Jul 2026 11:20:37 +0100
> Andrew Murray <amurray@thegoodpenguin.co.uk> wrote:
>
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 2884103b93bca7b76cd3a93946276074cf62d0a1..1118feda87b1b04543b1da0bd52c090b1fddaeac 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -650,11 +650,19 @@ Kernel parameters
> > See Documentation/block/cmdline-partition.rst
> >
> > boot_delay= [KNL,EARLY]
> > - Milliseconds to delay each printk during boot.
> > - Only works if GENERIC_CALIBRATE_DELAY is enabled,
> > - and you may also have to specify "lpj=". Boot_delay
> > - values larger than 10 seconds (10000) are assumed
> > - erroneous and ignored.
> > + Milliseconds to delay each printk during and post boot.
> > + Boot time delays only work if GENERIC_CALIBRATE_DELAY
> > + is enabled.
> > +
> > + Once booted the delay can be removed or adjusted via
> > + the printk_delay sysctl.
> > +
> > + Please note that you may also have to specify "lpj=".
> > + Boot_delay values larger than 10 seconds (10000) are
> > + assumed erroneous and ignored.
> > +
> > + This will soon be deprecated, please use printk_delay
> > + instead.
> > Format: integer
> >
> > bootconfig [KNL,EARLY]
> > @@ -5468,6 +5476,19 @@ Kernel parameters
> > printk.time= Show timing data prefixed to each printk message line
> > Format: <bool> (1/Y/y=enable, 0/N/n=disable)
> >
> > + printk_delay= [KNL,EARLY]
> > + Milliseconds to delay each printk during and post boot.
> > + Boot time delays only work if GENERIC_CALIBRATE_DELAY
> > + is enabled.
> > +
> > + Once booted the delay can be removed or adjusted via
> > + the printk_delay sysctl.
> > +
> > + Please note that you may also have to specify "lpj=".
> > + printk_delay values larger than 10 seconds (10000) are
> > + assumed erroneous and ignored.
> > + Format: integer
> > +
>
> The two parameters descriptions are identical. They should show something
> different. Is it just that the "boot_delay" name is getting deprecated?
Good point. We could remove all the details from the boot_delay
comment. Instead, we could say that it an obsolete alias to
printk_delay, will soon be drecated, ...
Best Regards,
Petr
^ permalink raw reply
* [PATCH v2 2/2] mm/zswap: Support batch writeback in shrink_memcg()
From: Hao Jia @ 2026-07-17 8:51 UTC (permalink / raw)
To: akpm, tj, hannes, shakeel.butt, mhocko, yosry, mkoutny, nphamcs,
chengming.zhou, muchun.song, roman.gushchin
Cc: linux-mm, linux-kernel, linux-doc, Hao Jia
In-Reply-To: <20260717085151.22822-1-jiahao.kernel@gmail.com>
From: Hao Jia <jiahao1@lixiang.com>
Currently, shrink_memcg() writes back at most one entry per-node during
its traversal. This makes shrink_worker() inefficient, as it must
repeatedly re-enter shrink_memcg() to make any substantial progress.
Under high memory pressure, this can cause the writeback speed to be
too slow to keep up with refaults, leading to zswap store failures and
forcing pages to skip zswap and go directly to disk, which results in
an LRU inversion.
To address this, extend shrink_memcg() and rewrite its LRU iteration logic
to support batch writeback. Introduce the nr_to_scan parameter to bound how
many pages are scanned per call. This enables batch writeback in the
shrink_worker() path, while maintaining a low scan budget in the
zswap_store() path.
Test Setup:
- Total memory: 32 GB.
- zswap settings: max_pool_percent=1, accept_threshold_percent=50,
shrinker_enabled=N.
Test Case 1:
Allocate 512MB of anonymous pages and fill them with random data (to avoid
compression), then use cgroup memory.reclaim to force a large amount of
anonymous pages into zswap. At an interval of 2ms, allocate a 4K anonymous
page where the first 4 bytes are random numbers and the rest are zeros, and
then trigger a reclamation of this 4K anonymous page through cgroup
memory.reclaim. When the pool threshold is reached, shrink_memcg() will
be triggered.
The test data after running for 120s is as follows:
Baseline Patched
shrink_worker wakeups 5,363 85
shrink_memcg calls 11,373,201 180,928
written_back pages 40,212 40,236
zswap_store calls 161,190 168,741
store succeeded (ret=1) 102,743 127,644
store rejected (ret=0) 58,447 41,097
store reject rate ~36% ~24%
pool_limit_hit delta 55,826 14,062
pswpout 98,659 81,333
pswpin 2 1
Test Case 2:
To consistently force zswap store failures and trigger shrink_worker(),
the following stress-ng command was run for 120 seconds within a cgroup
limited to a memory.max of 1G:
bash -c 'echo $$ > /sys/fs/cgroup/zswaptest/cgroup.procs ; \
exec stress-ng --vm 4 --vm-bytes 4G --vm-keep --vm-method rand-set -t \
120s -q'
The test data after running for 120s is as follows:
Baseline Patched
shrink_worker wakeups 5,640 987
shrink_memcg calls 8,481,500 2,504,818
written_back pages 260 768,576
zswap_store calls 2,742,756 2,301,414
store succeeded (ret=1) 934,640 1,308,686
store rejected (ret=0) 1,808,116 992,728
store reject rate ~66% ~43%
pool_limit_hit delta 1,181,310 101,593
pswpout 1,808,376 1,761,304
pswpin 4,288,497 3,902,658
Under identical workloads and runtimes, batching the zswap shrinker
exhibits a significant reduction in both shrink_worker wakeups and
shrink_memcg calls. Furthermore, the sharp drop in both pool_limit_hit
and zswap_store rejections demonstrates that batching the zswap shrinker
effectively mitigates zswap_store failures caused by hitting the pool
limit. This significantly prevents pages from bypassing zswap and falling
back directly to disk, thereby reducing LRU inversion.
Suggested-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Hao Jia <jiahao1@lixiang.com>
---
mm/zswap.c | 47 ++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 40 insertions(+), 7 deletions(-)
diff --git a/mm/zswap.c b/mm/zswap.c
index 48fc7b575e24..6a09b9ebfd25 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -1275,9 +1275,27 @@ static struct shrinker *zswap_alloc_shrinker(void)
return shrinker;
}
-static int shrink_memcg(struct mem_cgroup *memcg)
+#define NR_ZSWAP_WB_BATCH 64UL
+
+/*
+ * Scan up to @nr_to_scan pages across the per-node zswap LRUs of @memcg
+ * and write back the reclaimable ones.
+ *
+ * Since the second-chance algorithm rotates referenced entries to the
+ * LRU tail, the per-node scan is capped at the current LRU length so
+ * each entry is scanned at most once per call. It is up to the caller
+ * to handle retries, deciding whether to scan another memcg to complete
+ * the full iteration, or to rescan the current memcg to drain its zswap
+ * entries.
+ *
+ * Return: 0 if at least one entry was written back, -EAGAIN if entries
+ * were scanned but none could be written back, or -ENOENT if @memcg has
+ * writeback disabled, is a zombie cgroup, or has empty zswap LRUs.
+ */
+static int shrink_memcg(struct mem_cgroup *memcg, unsigned long nr_to_scan)
{
- int nid, shrunk = 0, scanned = 0;
+ unsigned long nr_remaining = nr_to_scan;
+ int nid, shrunk = 0;
if (!mem_cgroup_zswap_writeback_enabled(memcg))
return -ENOENT;
@@ -1290,14 +1308,29 @@ static int shrink_memcg(struct mem_cgroup *memcg)
return -ENOENT;
for_each_node_state(nid, N_NORMAL_MEMORY) {
- unsigned long nr_to_walk = 1;
+ unsigned long nr_to_walk;
+ /*
+ * Cap the scan at per-node LRU length so each entry is scanned
+ * at most once per call.
+ */
+ nr_to_walk = min(nr_remaining,
+ list_lru_count_one(&zswap_list_lru, nid, memcg));
+ if (!nr_to_walk)
+ continue;
+
+ nr_remaining -= nr_to_walk;
shrunk += list_lru_walk_one(&zswap_list_lru, nid, memcg,
&shrink_memcg_cb, NULL, &nr_to_walk);
- scanned += 1 - nr_to_walk;
+ /* Return the unused share of the budget to the pool. */
+ nr_remaining += nr_to_walk;
+
+ if (!nr_remaining)
+ break;
}
- if (!scanned)
+ /* Nothing was scanned: every LRU under @memcg was empty. */
+ if (nr_remaining == nr_to_scan)
return -ENOENT;
return shrunk ? 0 : -EAGAIN;
@@ -1369,7 +1402,7 @@ static void shrink_worker(struct work_struct *w)
goto resched;
}
- ret = shrink_memcg(memcg);
+ ret = shrink_memcg(memcg, NR_ZSWAP_WB_BATCH);
/* drop the extra reference */
mem_cgroup_put(memcg);
@@ -1493,7 +1526,7 @@ bool zswap_store(struct folio *folio)
objcg = get_obj_cgroup_from_folio(folio);
if (objcg && !obj_cgroup_may_zswap(objcg)) {
memcg = get_mem_cgroup_from_objcg(objcg);
- if (shrink_memcg(memcg)) {
+ if (shrink_memcg(memcg, 1)) {
mem_cgroup_put(memcg);
goto put_objcg;
}
--
2.34.1
^ permalink raw reply related
* [PATCH v2 1/2] mm/zswap: Fix global shrinker when memory cgroup is disabled
From: Hao Jia @ 2026-07-17 8:51 UTC (permalink / raw)
To: akpm, tj, hannes, shakeel.butt, mhocko, yosry, mkoutny, nphamcs,
chengming.zhou, muchun.song, roman.gushchin
Cc: linux-mm, linux-kernel, linux-doc, Hao Jia, stable
In-Reply-To: <20260717085151.22822-1-jiahao.kernel@gmail.com>
From: Hao Jia <jiahao1@lixiang.com>
Zswap writeback on hitting the pool limit is broken when memory cgroup
is disabled, because mem_cgroup_iter() always returns NULL. Therefore,
the global shrinker shrink_worker() always takes the !memcg branch.
After MAX_RECLAIM_RETRIES empty walks, the worker simply gives up, so it
fails to write back anything.
Therefore, when memory cgroup is disabled, fall through with the !memcg
branch and shrink the root memcg directly.
With memcg disabled, shrink_memcg() only returns -ENOENT when the root
LRU is empty, which means the total pages are already below thr. In the
absence of heavy concurrent zswap stores, the loop then safely bails out
via the zswap_total_pages() <= thr check; otherwise, it will resume
shrinking the memcg after processing the reschedule check. For any other
return value from shrink_memcg(), the loop is guaranteed to terminate,
either after MAX_RECLAIM_RETRIES failures or once the threshold is met.
Fixes: a65b0e7607cc ("zswap: make shrinking memcg-aware")
Cc: stable@vger.kernel.org
Suggested-by: Nhat Pham <nphamcs@gmail.com>
Acked-by: Nhat Pham <nphamcs@gmail.com>
Acked-by: Yosry Ahmed <yosry@kernel.org>
Reported-by: Yosry Ahmed <yosry@kernel.org>
Closes: https://lore.kernel.org/all/CAO9r8zPVzMKFbCixxD-qgtRrkFxWVrHiZZeLc=eyTPKPVQgX4g@mail.gmail.com
Signed-off-by: Hao Jia <jiahao1@lixiang.com>
---
mm/zswap.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/mm/zswap.c b/mm/zswap.c
index b5a17ea20237..48fc7b575e24 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -1356,11 +1356,12 @@ static void shrink_worker(struct work_struct *w)
} while (memcg && !mem_cgroup_tryget_online(memcg));
spin_unlock(&zswap_shrink_lock);
- if (!memcg) {
- /*
- * Continue shrinking without incrementing failures if
- * we found candidate memcgs in the last tree walk.
- */
+ /*
+ * A NULL memcg ends a full hierarchy pass (except when memcg is
+ * disabled, where it is always NULL: fall through to the root LRU).
+ * Count a failure only if the last pass found no candidates.
+ */
+ if (!memcg && !mem_cgroup_disabled()) {
if (!attempts && ++failures == MAX_RECLAIM_RETRIES)
break;
@@ -1379,7 +1380,7 @@ static void shrink_worker(struct work_struct *w)
* and failures.
*/
if (ret == -ENOENT)
- continue;
+ goto resched;
++attempts;
if (ret && ++failures == MAX_RECLAIM_RETRIES)
--
2.34.1
^ permalink raw reply related
* [PATCH v2 0/2] mm/zswap: Fixes and improves the zswap global shrinker
From: Hao Jia @ 2026-07-17 8:51 UTC (permalink / raw)
To: akpm, tj, hannes, shakeel.butt, mhocko, yosry, mkoutny, nphamcs,
chengming.zhou, muchun.song, roman.gushchin
Cc: linux-mm, linux-kernel, linux-doc, Hao Jia
From: Hao Jia <jiahao1@lixiang.com>
This series fixes and improves the zswap global shrinker (shrink_worker()):
Patch 1: Fix missing global shrinker when memory cgroup is disabled.
Patch 2: Extend shrink_memcg() to support batch writeback and update its
return value semantics, thereby improving the writeback efficiency
in the shrink_worker() path.
v1->v2:
- Add a reschedule check to the -ENOENT return path in shrink_memcg() to
handle the theoretical issue of prolonged heavy concurrent zswap stores.
- Remove the shrink_memcg() return value changes part, and include a more
detailed test report in the commit message.
[v1] https://lore.kernel.org/all/20260714081510.16895-1-jiahao.kernel@gmail.com
Hao Jia (2):
mm/zswap: Fix global shrinker when memory cgroup is disabled
mm/zswap: Support batch writeback in shrink_memcg()
mm/zswap.c | 60 ++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 47 insertions(+), 13 deletions(-)
--
2.34.1
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox