From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from OSPPR02CU001.outbound.protection.outlook.com (mail-norwayeastazon11013052.outbound.protection.outlook.com [40.107.159.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 656E8368D66; Sat, 12 Sep 2026 11:35:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.159.52 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789212925; cv=fail; b=Zi7rA7SDW2ZPUox3b6JCrolS4opw4pfUWnqlpwZ7X3OFQiCNqINkfuTz/pmxjI59rPm5pc8pmoRQVm71HrqZI8QLNUlTmeTrcDVwi2PQ4CxGkzYZ1KtWiJ844ISZeJYQV6y5dD89MA6Lqs+SBoXYPxjsZXvAFiXdTwRaXKmq1y4= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789212925; c=relaxed/simple; bh=aNuysAIZK+SV0keyEht0Xa6QvaLJXjWZQ5hkn+DCEqM=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=hktqC/WXhB2ZK77z+3YORK+kLt7PKwqOGmB3wJlyYNfFoUTJ4nQX576z11Hdnt9tvgHl6AazAigiICMHVruy0+1ifr5R4k6IN6J7onWeRMoWlitUdeh0Eg+MO7e8XYw2ejzc58zIAfpz6Imhz1qsKXQ9m8mEIWt9nehJo+5AucA= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oss.nxp.com; spf=pass smtp.mailfrom=oss.nxp.com; dkim=pass (2048-bit key) header.d=NXP1.onmicrosoft.com header.i=@NXP1.onmicrosoft.com header.b=u/fND1jP; arc=fail smtp.client-ip=40.107.159.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oss.nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.nxp.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=NXP1.onmicrosoft.com header.i=@NXP1.onmicrosoft.com header.b="u/fND1jP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=v2uXDY6eZPSLXsCWx2FGoxlWq1dh9j7QlT/yjYitpmDFRdXgHjuA811zOumfSZGL5ICiwV7EJuLfFHovztzcbKiOUxPzkfTlmH2RYmYGdDkEX2uqL6FKHDoRAf4cK0lnjvfHXZvVbBq2wc0qwdMIJoRAal0+YThWyhWXuLo91jYsL9aiEuH5B/vBVJw8LXjXl5TQVRuksq5X1sh3jCdKxIBVxA4LxMfLHHAzSJ+OgLb4ol/ZoQJs0IZAGA1LLxHIkzhUPe3Iw/bWX453NclRkoENY3egEVupzutKHULXvbj+z9CqhA3SzJhMQ+ueIuDNWWGuKUQlWetukvP7zpG1aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4xE0TSLzwG/xX5tWtPHDZ8kLfK4kpvfFTJNGmr9cxW8=; b=En8tLANOngCd/vLncEr2aw1haYRSz7uWjHaVMfmqM8jKgaPN/xzrs6RVV/ZiWIfWcvq/PKir3KGhuPVkxEhM2VS/KIE4rAaNPwYnINT2AyprZfQhzgibEnbCP9o82Yg5TIMYUBvXOMlHbcQB+5xtQJzFeaHTwwfUyD3FgFEtbKipShbPQnEF8FgngRdK3lMZJI1BJvFCYbEFQ5rtGNdYgCn1ELLWdPjdXH7hcoMp0JxKs2a9aY9iz8YoPNsL6I/XOtZJKwXpRfEFlxRHO7chr4w7n4jS0dP6kG8DH+tgdPaA8YYd1UCcMlGNZnox3ntb5qW8SLBnFNPLfWupDncpyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector1-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4xE0TSLzwG/xX5tWtPHDZ8kLfK4kpvfFTJNGmr9cxW8=; b=u/fND1jPU0RWRuRUsv7ysEKlbHzEfx5KuZ3gxu1n5yLcnHsLfb3Va0vVc7gsUifBp6TiFWGucwl5PhKIvZVtYASerMBMl7BnnSTjCuhaPnasG2o/TbYxIO29vPiC5gQlv6dESQwdf6HBFrhUQxE+S12LrIURJKMA6pT7oDyHgxy/spY8JJ9ehdRKZC8NuzkCPsDg2s8YL8Xp36X8HJ3NZLLUXaQEK2UxSorhIjEtUN8V5Il7XEy2vrlOx7yuUv5HGGqeagCjnxicZJJznwM5nB0wLeXATdP8g0uISqFQS0o31NQ5l7A9jvSDmWrg5SwBsbf5qjaG2Ihbk4hfAJ2qhg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from AM9PR04MB8469.eurprd04.prod.outlook.com (2603:10a6:20b:414::15) by ZR6PR04MB466071.eurprd04.prod.outlook.com (2603:10a6:910:dc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.11; Sat, 12 Sep 2026 11:34:54 +0000 Received: from AM9PR04MB8469.eurprd04.prod.outlook.com ([fe80::1f31:d3d0:6150:b49c]) by AM9PR04MB8469.eurprd04.prod.outlook.com ([fe80::1f31:d3d0:6150:b49c%3]) with mapi id 15.21.0406.007; Sat, 12 Sep 2026 11:34:54 +0000 From: pankaj.gupta@oss.nxp.com Date: Sat, 12 Sep 2026 22:33:39 +0530 Subject: [PATCH v50 5/7] firmware: imx: adds miscdev Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Message-Id: <20260912-imx-se-if-v50-5-80834ef510d3@nxp.com> References: <20260912-imx-se-if-v50-0-80834ef510d3@nxp.com> In-Reply-To: <20260912-imx-se-if-v50-0-80834ef510d3@nxp.com> To: Jonathan Corbet , Shuah Khan , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Frank Li , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , Pankaj Gupta , Randy Dunlap Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, imx@lists.linux.dev, linux-arm-kernel@lists.infradead.org X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1789232651; l=172833; i=pankaj.gupta@nxp.com; s=20260817; h=from:subject:message-id; bh=EHy+qdwioTazLjL3JSxU2kBn9SI81itILjgyQtAwbXk=; b=Z2j9WArnGej/4MebsnH0dw5/SW/YAi6KGtE0hkUzPQ6sPV6UmJsmYd2K/3w7R1DTf3tiZJkrE Pyv7qg4yJzDAuRVm41mD9HwjqlHmpZJ3y2gIpXdrjl0k2cxdkIsbhFI X-Developer-Key: i=pankaj.gupta@nxp.com; a=ed25519; pk=g4ZgzIWbpXnSxRsoH+l6PWLP78a+Mzpl8e6RQe74d5Y= X-ClientProxiedBy: MA5PR01CA0078.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a01:1ad::11) To AM9PR04MB8469.eurprd04.prod.outlook.com (2603:10a6:20b:414::15) Precedence: bulk X-Mailing-List: devicetree@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR04MB8469:EE_|ZR6PR04MB466071:EE_ X-MS-Office365-Filtering-Correlation-Id: 451419be-2637-4a6d-dff4-08df10c1dd3d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|366016|1800799024|19092799006|7416014|376014|5023799004|11063799006|56012099006|6133799003|7136999003|3023799007|10067099003|22082099003|18002099003|11062099010|13003099007|921020; X-Microsoft-Antispam-Message-Info: 4q/NQ0+xOGcG7TuinHrXgCPI7+mcQLE8eH1bovVdmKeXObYv58gABGsUITinqBn2Tbfel4cxGwXa/8gR33oLsB7ht92F4cLsgxA1yRKA0leWTfxskOmzVJXB0+A9MxeX7rxIz8HBbkZ1KviezOtMKhFmlsRUOX5seTvABO9hj00QDJMfbDthofkSs0g6EUEkPSIckj8CA0MHCcPNWkXZv+bGFRbvO5n4hjHbYAAdUVdzdLJGNJXR0/iVC+MJefiomFM0EpbnBN0JDRAgl107ys5h4WMrZRUClIB80qWB9GxqmE78AhPVcb717D0z7/ICdsfLhjdreGsVwRqamo+7MWimGSPUYCQLhqrhZqo/1RFnS987zjcRNmYIPMkFAIHmku5/81Oz/e/V3/rxrSK9U6YyVtnmJIrOOOgVeQ/M+Ol2GXb9Tgjw7LqiIqVKoDvAhBDEVkL9lxOn3Pyxzu3GhQ272LDK/cRdebQ5dl0V5HX2wAo+4jloGQH2RjWpn2iE6XFDzGwg7cIEGdzj/If1QhUjKG8XOsOaZpgdCkP3KFwilt0bpZ+KqyUrMd16OSX0kZ9+2pBTBVLsiYvZ1iZspAfUp2lXVPudGw71bBn46p1doFDPHaCQvFRTg5b0l51X39wsllibQsbMWiQliUGVxWGc3+JKHQ+BYSGi5K5vEe1aX/s4AkN4b+s/mySBES4pjORumHQ0ME8SPfOs8grXXA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR04MB8469.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(366016)(1800799024)(19092799006)(7416014)(376014)(5023799004)(11063799006)(56012099006)(6133799003)(7136999003)(3023799007)(10067099003)(22082099003)(18002099003)(11062099010)(13003099007)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RURJR3lQQ3VIL29nVnpXdVNLUHBscWRZTXhtZ3o5ekpZK1ROUE1ZektzR2NK?= =?utf-8?B?enJzWEpOUGdybWxQTkgwUTg0ZUZ5Lzg3V0lCaWdmeDZaQnBJMXRhZWZRSkpD?= =?utf-8?B?Mis0bnVSTGtEeVBZWkJ3MlR4Zjc4SkVveWFYdEwvbUpHMGpEZ1JmUFEvUkJt?= =?utf-8?B?c0phemM3MHk4eWxlMXl2U2J2aXIzN0NVZjM4N2VqRmFhbStJUFNXOUt3d012?= =?utf-8?B?bXNuUDMxNjdoOVdIWlVIZlUvT1ZHU2k5SzJ1b3ZzZUluMitxS0ZFTHFFS2FR?= =?utf-8?B?TDhsb2hod2kyOE5seXFVaFIrcFJrUjVYQjRWQWtDbmxZN3E0SEdITjJ0eExt?= =?utf-8?B?c2VYSndGMTR1cWFTMzdhZTNjaTVtbzFkeko1V3g0T0U0Zjdqb1N5bWJha0N2?= =?utf-8?B?WTBUVDZhZm1JV09QUU5JREpvZjV6M3VNaFV2WlVBaVprTFFwVkRUU1BFbllC?= =?utf-8?B?WXZiL202b3RzOFJzaTBYRUpDVHI3T0JsdWFCRElENllCTGRmaDZ6dnhrb2dK?= =?utf-8?B?ZHhtZ2pBcTBQaEg1TCtBU0ZwTkYrdUd0LzRGcVJNVnZxZ20rQzgxVHhzeXZ3?= =?utf-8?B?cG8rSWYzVUlzUWVXd2tIOGZnZE1jQ0hCNzRtOVo0STFwdk4xbEJvSHVOc1ox?= =?utf-8?B?d21GV3hYQVFPTTJWY3JHME1CUUhPSlVhcjRETW96K1RTbkt1aEtJWmZNbCtn?= =?utf-8?B?Q3lPSTR0a21RL0VvczR5QUdoa3oyWGhob0ZPT2JZdDNYdU9uQlhDcHlURDdw?= =?utf-8?B?QmgwaW9qUUI2Tm5qdTBhVnpVQ0o4TnZ5aTloWVZVNEJzVEFyV1RhTVpSbGhJ?= =?utf-8?B?ZHM4aVNBVjdWVnRCSzhvdHAyd3ZRUm1FZDRIR3NtS0lvenZyY2NEQnFvd0hw?= =?utf-8?B?NURZNDUxMW8rZ2hKNW5vUHp4S3NSTEExbFZ1YWtJQzkza2lFSWhQZ1h6V1Vt?= =?utf-8?B?UWNHaHRNK1BEWW9JZE1MeHlNRmVOaExFN0lzN2I2MlNydS8xeWtZLzBYNmlN?= =?utf-8?B?d1dhN3FmQ2dyYXR4SzdPMEF0bFdVWXdxUktCUVA5WHVTUFV2RGk4LzNnclRu?= =?utf-8?B?ZWpFbkJWTjhPd0hvNnFCaStoUVgxTnFlZXZKSzRYNXQ0Sk10RnJFdnU3dlpm?= =?utf-8?B?cFZBMy9hMkpTZ2xDRlJoWjlvdWMzZzU0NFBwKzErenM0NEZIaTR5TXhadFly?= =?utf-8?B?S2ZqQnVYVS92WjJqS2hjVUduaUl6ZkVtRWN0WmJiVjFvNnFGeWErcHZkTkpM?= =?utf-8?B?NzE5WDc0dlhHMzNoeXZRdkNEN1p1aHpoYnlHSk1FZzROUStTUkttMHNkL21n?= =?utf-8?B?Nk5Fd0t1d1NGd3JOMGpGMXFsODdUbFNsWE9XdTFTRGdFY2R6eVUrVEI5dGRo?= =?utf-8?B?SWgvdzE1YTVTODZESzRmdENRaERPYnc0OGluNm80WUJlODYxMWJrS3JyVVM1?= =?utf-8?B?dzQ3WmVBVURZSDErR2tRNXVXRDcvODFRRk1KYlNma3p0dnpaN25HLy9kUGdK?= =?utf-8?B?TnFRZVBNM0ZlUnhoMGxpZlJVVVVCVm0vbVlXVTJLRVZBZS91ZFVVTXlQUzFH?= =?utf-8?B?dDY4enhiVWFiMXpYdENvbCtuZnBkU0tYZFIzOXR5V2E5a0Q1NHlEUHhKNzh1?= =?utf-8?B?UThGcUx1bktkVmtDSnpWaUVvVkxpMFR5eGQxeVl6citUam8yWE1uT1JMYjdz?= =?utf-8?B?aGJCa2l4cGhoS2VlR0loalZqQjBqMWJpQ3J5YUVOQVNFbkhpemo4WjFVd0pP?= =?utf-8?B?amV3a3lkaG5xbHpqVTVFeVJrYVRLRzVaQjB3NFVLUTVZbDJRMVRBcFpHWE15?= =?utf-8?B?S01MOEo1QUMrUlJxbjlmQTl2OXQ1RnNac1F2SmZudmdsZStRUjhSWndra2RF?= =?utf-8?B?NDRRU2RxazR6MXA1clV2N2dxVHFlYnJEemxxdEppamY0eEJSak85SytCUkJD?= =?utf-8?B?ZSt0M3F4QnhNWlJuZ1VXWG1HZXROdzdUQlpTdElzNEo0SnNwd1dNYnk5bHhJ?= =?utf-8?B?aEJtSUswd2hra2lHOHBzbmFwSVhzRmlxeVhlK1J3dXNlWTJDMThqcUFWNjBI?= =?utf-8?B?aDZseCt5cDBFVWtoN2VhbU5CS2dGWDBLc2pYaVF5MW55TVJmSXdyVGVhL08w?= =?utf-8?B?OStqZjhpVWQxQVI5N3ZYYzNIVXZyZklVTXRpWkdBZWRpaEhGUk93eTNjTmMy?= =?utf-8?B?Z3JSeUNUNytOVGtZUm8rTlo2QTFuL2s5VVgyUmdjRXVqQmZISnF6Z1Y4d1Uv?= =?utf-8?B?RFJxZFB2TUExWWQyU3B0VVI4bFZCdkpESnNMN2VXa2VCNDFEV0RnWDdBM21Z?= =?utf-8?B?YlFjMC9ZTWMvQ2trdnFTQUFVcXZ5dGg3QVMrUWNQanVrcmI2R29Xd3VRbWQz?= =?utf-8?Q?UUVCIoDSvAXPOpyFY2H33w3cAjuCUlV0rlYoW?= X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 451419be-2637-4a6d-dff4-08df10c1dd3d X-MS-Exchange-CrossTenant-AuthSource: AM9PR04MB8469.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Sep 2026 11:34:54.2024 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +RTlFVlja3eDI/71pN43yOIhXDLo+G53DGWPOPTD/CYj4pJGF2ycL1JOrAvb1v7qTvPO+eAZN9RD3yQzmZuwO6tFl84U5r8R4Uu8bH1eKl0uUdHrdib35jnn8UJTNIHC X-MS-Exchange-Transport-CrossTenantHeadersStamped: ZR6PR04MB466071 From: Pankaj Gupta Adds the driver for communication interface to secure-enclave, that enables exchanging messages with NXP secure enclave HW IP(s) like EdgeLock Enclave, from: - User-Space Applications via character driver. ABI documentation for the NXP secure-enclave driver. User-space library using this driver: - i.MX Secure Enclave library: -- URL: https://github.com/nxp-imx/imx-secure-enclave.git, - i.MX Secure Middle-Ware: -- URL: https://github.com/nxp-imx/imx-smw.git Following checks are performed on the incoming msg-header, to block exchanging invalid arbitrary commands: - maximum allowed words, - check if command-tag & response-tag are valid - version, - command id validation check, to allow limited base-line API(s) and restrict following: - exchanging power management commands. - reset requests. - BBSM configuration requests. - re-initializing the FW. - RNG init - CAAM resource release management - SE's internal memory management. from user-space. Signed-off-by: Pankaj Gupta --- Changes from v49 to v50: - Introduce an explicit gating scheme in ele_msg_send_rcv() so that the send+receive path reacts correctly to driver teardown and to the firmware-busy circuit breaker, and add an exclusive messaging-interface reservation that lets the two privileged recovery flows resynchronise session/storage state with firmware without being rejected by that same circuit breaker. - Two independent gates are checked under clbk_rx_lock at the top of ele_msg_send_rcv(), before a new transaction is armed: - going_away (set-once teardown barrier): a caller racing unbind is rejected with the non-retryable -ENODEV. going_away is checked before fw_busy so a caller never gets a misleading retryable -EBUSY when the device is actually gone for good. - fw_busy (circuit breaker): while set, new transactions are rejected with the retryable -EBUSY. - Harden DMA buffer bounds checking in se_val_cmd_addrs(). Rework the size- validation logic as an explicit switch (f->size_idx) that resolves the buffer length from one of these sources; every embedded address is bound- checked and the driver never forwards one without a length it can verify: - SE_CMD_ADDR_FIXED_SIZE: the length is a firmware-defined literal supplied in buf_size. A zero buf_size is rejected with -EINVAL, and the whole buffer [addr, addr + buf_size) must fit inside the window (-EACCES otherwise). The two buffers whose length is a fixed firmware constant - GEN_KEY_BLOB load_address (0x30) and AUTH_ENC_NEW iv_address_out (12) - use this sentinel so they keep a real end-bound check. - SE_CMD_RCVR_ADDR_VAR_SIZE: the length is taken from se_if_priv.cmd_rcvr_var_size, supplied by firmware in the preceding export command (command-receiver responses). It keeps the same overflow-safe "len > end - addr" comparison. - SE_CMD_ADDR_DEDUCE_SZ: no length word is carried in the payload. Instead the buffer begins with a struct fw_tag_len_vers_info header whose length field gives the total buffer byte count, which is read to bound the whole buffer. Only two genuinely addr-only buffers (OEM_AUTH_CONTAINER container header, KEYSTORE_REPROV_ENABLE signed message) use this type. - payload-word index (the default): size_idx names a payload word carrying the length, extracted with size_shift/size_mask. It keeps the same overflow-safe "len > end - addr" comparison (addr is already known to be >= base and < end). A size_idx that names an out-of-range word, or a zero size_mask, is treated as a descriptor construction bug and rejected with -EINVAL. (SE_CMD_ADDR_ALWAYS is not a size source; it is used in flag_idx to mark a word that is always a DMA address rather than a flag-gated key id.) Net effect: every descriptor that can express a length is fully bounded, and no descriptor can express an address without one. --- Documentation/ABI/testing/se-cdev | 45 + drivers/firmware/imx/Makefile | 2 +- drivers/firmware/imx/ele_base_msg.c | 113 +- drivers/firmware/imx/ele_base_msg.h | 19 + drivers/firmware/imx/ele_common.c | 443 ++++++- drivers/firmware/imx/ele_common.h | 104 +- drivers/firmware/imx/ele_fw_api.c | 404 ++++++ drivers/firmware/imx/ele_fw_api.h | 104 ++ drivers/firmware/imx/ele_msg_addr_field.c | 619 ++++++++++ drivers/firmware/imx/se_ctrl.c | 1924 ++++++++++++++++++++++++++++- drivers/firmware/imx/se_ctrl.h | 175 ++- include/uapi/linux/se_ioctl.h | 97 ++ 12 files changed, 3965 insertions(+), 84 deletions(-) diff --git a/Documentation/ABI/testing/se-cdev b/Documentation/ABI/testing/se-cdev new file mode 100644 index 000000000000..0a353caeaab2 --- /dev/null +++ b/Documentation/ABI/testing/se-cdev @@ -0,0 +1,45 @@ +What: /dev/_mu[0-9]+_ch[0-9]+ +Date: Mar 2025 +KernelVersion: 6.8 +Contact: linux-imx@nxp.com, pankaj.gupta@nxp.com +Description: + NXP offers multiple hardware IP(s) for secure enclaves like EdgeLock- + Enclave(ELE), SECO. The character device file descriptors + /dev/_mu*_ch* are the interface between userspace NXP's secure- + enclave shared library and the kernel driver. + + The ioctl(2)-based ABI is defined and documented in + [include] + ioctl(s) are used primarily for: + + - shared memory management + - allocation of I/O buffers + - getting mu info + - setting a dev-ctx as receiver to receive all the commands from FW + - getting SoC info + - send command and receive command response + + The following file operations are supported: + + open(2) + Currently the only useful flags are O_RDWR. + + read(2) + Every read() from the opened character device context is waiting on + wait_for_completion_interruptible_timeout, that gets set by the + registered mailbox callback function, indicating a message received + from the firmware on message-unit. + + write(2) + Every write() to the opened character device context needs to acquire + `fops_lock` + `se_if_cmd_lock` lock before sending message on to the + message unit. + + close(2) + Stops and frees up the I/O contexts that were associated + with the file descriptor. + +Users: https://github.com/nxp-imx/imx-secure-enclave.git, + https://github.com/nxp-imx/imx-smw.git, + crypto/skcipher, + drivers/nvmem/imx-ocotp-ele.c diff --git a/drivers/firmware/imx/Makefile b/drivers/firmware/imx/Makefile index 4412b15846b1..33f30eaedad5 100644 --- a/drivers/firmware/imx/Makefile +++ b/drivers/firmware/imx/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_IMX_SCU) += imx-scu.o misc.o imx-scu-irq.o rm.o imx-scu-soc.o obj-${CONFIG_IMX_SCMI_CPU_DRV} += sm-cpu.o obj-${CONFIG_IMX_SCMI_MISC_DRV} += sm-misc.o obj-${CONFIG_IMX_SCMI_LMM_DRV} += sm-lmm.o -sec_enclave-objs = se_ctrl.o ele_common.o ele_base_msg.o +sec_enclave-objs = se_ctrl.o ele_common.o ele_base_msg.o ele_fw_api.o ele_msg_addr_field.o obj-${CONFIG_IMX_SEC_ENCLAVE} += sec_enclave.o diff --git a/drivers/firmware/imx/ele_base_msg.c b/drivers/firmware/imx/ele_base_msg.c index 650153d39f46..b60665120db2 100644 --- a/drivers/firmware/imx/ele_base_msg.c +++ b/drivers/firmware/imx/ele_base_msg.c @@ -15,13 +15,66 @@ #define FW_DBG_DUMP_FIXED_STR "ELE" -static void ele_get_info_cleanup(struct se_if_priv *priv, u32 *buf, dma_addr_t d_addr, - size_t size) +int ele_uapi_allowed_base_cmd(struct se_if_device_ctx *dev_ctx, + struct se_msg_hdr *header, u32 tx_msg_sz) { - if (priv->mem_pool) - gen_pool_free(priv->mem_pool, (unsigned long)buf, size); - else - dma_free_coherent(priv->dev, size, buf, d_addr); + struct se_api_msg *msg = container_of(header, struct se_api_msg, header); + const struct se_cmd_addr_field *fields; + size_t count; + + /* + * Identify the command first. Only commands in this allow-list may be + * issued from userspace; everything else is rejected. Once a command is + * known to be supported, decide whether it needs a DMA-address boundary + * check and, if so, run it before returning. + */ + switch (header->command) { + case ELE_PING_REQ: + case ELE_DEBUG_DUMP_REQ: + case ELE_OEM_VERIFY_IMAGE_REQ: + case ELE_OEM_REL_CONTAINER_REQ: + case ELE_FW_LIFE_CYCLE_REQ: + case ELE_READ_FUSE_REQ: + case ELE_GET_FW_VERS_REQ: + case ELE_RETURN_LIFE_CYCLE_REQ: + case ELE_GET_EVENT_REQ: + case ELE_COMMIT_REQ: + case ELE_GET_FW_STATUS_REQ: + case ELE_WRITE_FUSE: + case ELE_WRITE_SHADOW_FUSE_REQ: + case ELE_READ_SHADOW_FUSE_REQ: + return 0; + default: + /* Base commands that embed DMA addresses. */ + fields = ele_base_cmd_addr_fields(header->command, &count); + if (!count) + return -EOPNOTSUPP; + return se_val_cmd_addrs(dev_ctx, msg, tx_msg_sz, fields, count); + } +} + +static void ele_get_info_cleanup(struct se_if_priv *priv) +{ + /* For the case when priv->mem_pool != NULL: + * + * If this probe-time transaction timed out, the firmware may + * still write into the SRAM buffer after this function returns. + * Do not release it back to the pool while the firmware-busy + * circuit breaker still marks this context as owning an + * outstanding transaction. The buffer is reclaimed with the + * device on unbind; leaking this fixed-size probe buffer is + * preferable to letting the firmware corrupt reused pool memory. + * This mirrors the guard already applied on the shared-memory + * cleanup path below. + */ + + if (priv->mem_pool) { + if (se_is_fw_busy_ctx(priv->priv_dev_ctx)) + return; + se_cleanup_mem_pool_buf(priv->priv_dev_ctx, true); + } else { + se_dev_ctx_shared_mem_cleanup(priv->priv_dev_ctx); + } } /** @@ -45,6 +98,7 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info) if (!priv) return -EINVAL; + guard(mutex)(&priv->priv_dev_ctx->fops_lock); memset(s_info, 0x0, sizeof(*s_info)); struct se_api_msg *tx_msg __free(kfree) = @@ -58,24 +112,23 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info) return -ENOMEM; get_info_len = ELE_GET_INFO_BUFF_SZ; - if (priv->mem_pool) - get_info_data = gen_pool_dma_alloc(priv->mem_pool, - get_info_len, - &get_info_addr); - else - get_info_data = dma_alloc_coherent(priv->dev, - get_info_len, - &get_info_addr, - GFP_KERNEL); - if (!get_info_data) { - dev_err(priv->dev, - "%s: Failed to allocate get_info_addr.\n", __func__); - return -ENOMEM; + if (priv->mem_pool) { + ret = se_get_mem_pool_buf(priv->priv_dev_ctx, &get_info_data, + &get_info_addr, get_info_len); + if (ret) { + dev_err(priv->dev, "Failed[0x%x] to alloc from gen_pool.\n", ret); + return -ENOMEM; + } + } else { + ret = get_shared_mem_slot(priv->priv_dev_ctx, + &get_info_len, &get_info_addr, + &get_info_data); + if (ret) { + dev_err(priv->dev, "Failed to allocate buffer.\n"); + return -ENOMEM; + } } - /* gen_pool_dma_alloc() does not zero the buffer. */ - memset(get_info_data, 0, get_info_len); - se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header, ELE_GET_INFO_REQ, ELE_GET_INFO_REQ_MSG_SZ, true); @@ -84,9 +137,9 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info) tx_msg->data[2] = sizeof(*s_info); ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_GET_INFO_REQ_MSG_SZ, - rx_msg, ELE_GET_INFO_RSP_MSG_SZ); + rx_msg, ELE_GET_INFO_RSP_MSG_SZ, NULL); if (ret < 0) { - ele_get_info_cleanup(priv, get_info_data, get_info_addr, get_info_len); + ele_get_info_cleanup(priv); return ret; } @@ -94,13 +147,13 @@ int ele_get_info(struct se_if_priv *priv, struct ele_dev_info *s_info) ELE_GET_INFO_RSP_MSG_SZ, priv->if_defs->base_api_ver); if (ret < 0) { - ele_get_info_cleanup(priv, get_info_data, get_info_addr, get_info_len); + ele_get_info_cleanup(priv); return ret; } memcpy(s_info, get_info_data, sizeof(*s_info)); - ele_get_info_cleanup(priv, get_info_data, get_info_addr, get_info_len); + ele_get_info_cleanup(priv); return ret; } @@ -146,7 +199,7 @@ int ele_ping(struct se_if_priv *priv) ELE_PING_REQ, ELE_PING_REQ_SZ, true); ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_PING_REQ_SZ, - rx_msg, ELE_PING_RSP_SZ); + rx_msg, ELE_PING_RSP_SZ, NULL); if (ret < 0) return ret; @@ -205,7 +258,7 @@ int ele_service_swap(struct se_if_priv *priv, return -EINVAL; ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_SERVICE_SWAP_REQ_MSG_SZ, - rx_msg, ELE_SERVICE_SWAP_RSP_MSG_SZ); + rx_msg, ELE_SERVICE_SWAP_RSP_MSG_SZ, NULL); if (ret < 0) return ret; @@ -262,7 +315,7 @@ int ele_fw_authenticate(struct se_if_priv *priv, dma_addr_t contnr_addr, tx_msg->data[2] = lower_32_bits(img_addr); ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_FW_AUTH_REQ_SZ, rx_msg, - ELE_FW_AUTH_RSP_MSG_SZ); + ELE_FW_AUTH_RSP_MSG_SZ, NULL); if (ret < 0) return ret; @@ -311,7 +364,7 @@ int ele_debug_dump(struct se_if_priv *priv) memset(rx_msg, 0x0, ELE_DEBUG_DUMP_RSP_SZ); ret = ele_msg_send_rcv(priv->priv_dev_ctx, tx_msg, ELE_DEBUG_DUMP_REQ_SZ, - rx_msg, ELE_DEBUG_DUMP_RSP_SZ); + rx_msg, ELE_DEBUG_DUMP_RSP_SZ, NULL); if (ret < 0) return ret; diff --git a/drivers/firmware/imx/ele_base_msg.h b/drivers/firmware/imx/ele_base_msg.h index 02525d5e2873..50e2a1a75716 100644 --- a/drivers/firmware/imx/ele_base_msg.h +++ b/drivers/firmware/imx/ele_base_msg.h @@ -29,6 +29,19 @@ #define ELE_DEBUG_DUMP_REQ_SZ 0x4 #define ELE_DEBUG_DUMP_RSP_SZ 0x5c +#define ELE_OEM_AUTH_CONTAINER_REQ 0x87 +#define ELE_OEM_VERIFY_IMAGE_REQ 0x88 +#define ELE_OEM_REL_CONTAINER_REQ 0x89 +#define ELE_FW_LIFE_CYCLE_REQ 0x95 +#define ELE_READ_FUSE_REQ 0x97 +#define ELE_GET_FW_VERS_REQ 0x9d +#define ELE_RETURN_LIFE_CYCLE_REQ 0xa0 +#define ELE_GET_EVENT_REQ 0xa2 +#define ELE_COMMIT_REQ 0xa8 +#define ELE_GEN_KEY_BLOB_REQ 0xaf +#define ELE_GET_FW_STATUS_REQ 0xc5 +#define ELE_WRITE_FUSE 0xd6 + #define ELE_GET_INFO_REQ 0xda #define ELE_GET_INFO_REQ_MSG_SZ 0x10 #define ELE_GET_INFO_RSP_MSG_SZ 0x08 @@ -71,6 +84,10 @@ struct ele_dev_info { #define ELE_GET_INFO_BUFF_SZ (sizeof(struct ele_dev_info) \ + ELE_DEV_INFO_EXTRA_SZ) +#define ELE_DEV_ATTEST_REQ 0xdb +#define ELE_WRITE_SHADOW_FUSE_REQ 0xf2 +#define ELE_READ_SHADOW_FUSE_REQ 0xf3 + #define ELE_SERVICE_SWAP_REQ 0xdf #define ELE_SERVICE_SWAP_REQ_MSG_SZ 0x18 #define ELE_SERVICE_SWAP_RSP_MSG_SZ 0x0c @@ -97,4 +114,6 @@ int ele_service_swap(struct se_if_priv *priv, dma_addr_t addr, int ele_fw_authenticate(struct se_if_priv *priv, dma_addr_t contnr_addr, dma_addr_t img_addr); int ele_debug_dump(struct se_if_priv *priv); +int ele_uapi_allowed_base_cmd(struct se_if_device_ctx *dev_ctx, + struct se_msg_hdr *header, u32 tx_msg_sz); #endif diff --git a/drivers/firmware/imx/ele_common.c b/drivers/firmware/imx/ele_common.c index 838d2f957e74..2e3d7e104f09 100644 --- a/drivers/firmware/imx/ele_common.c +++ b/drivers/firmware/imx/ele_common.c @@ -5,6 +5,217 @@ #include "ele_base_msg.h" #include "ele_common.h" +#include "ele_fw_api.h" +#include "se_ctrl.h" + +int se_chk_tx_rsp_msg_hdr(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz) +{ + struct se_if_priv *priv = dev_ctx->priv; + + if (!header->size || header->size > MAX_WORD_SIZE) + return -EINVAL; + + if (header->tag != priv->if_defs->rsp_tag) + return -EINVAL; + + if (header->ver == priv->if_defs->base_api_ver) + return -EINVAL; + + else if (header->ver == priv->if_defs->fw_api_ver) + return ele_uapi_allowed_fw_rsp(dev_ctx, header, tx_msg_sz); + + return -EINVAL; +} + +int se_chk_tx_cmd_msg_hdr(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz, u32 rx_msg_sz) +{ + struct se_if_priv *priv = dev_ctx->priv; + + if (!header->size || header->size > MAX_WORD_SIZE) + return -EINVAL; + + if (header->tag != priv->if_defs->cmd_tag) + return -EINVAL; + + if (header->ver == priv->if_defs->base_api_ver) + return ele_uapi_allowed_base_cmd(dev_ctx, header, tx_msg_sz); + else if (header->ver == priv->if_defs->fw_api_ver) + return ele_uapi_allowed_fw_cmd(dev_ctx, header, tx_msg_sz, rx_msg_sz); + + return -EINVAL; +} + +/* + * Reject a command that embeds a DMA physical address which does not point + * inside this context's shared-memory window. The userspace library stages all + * command buffers in that coherent region (see get_shared_mem_slot), so any + * address outside [dma_addr, dma_addr + size) is not one the driver handed out + * and must not be forwarded to firmware. Absent optional buffers are encoded as + * a zero address and skipped; polymorphic key words are only range-checked when + * their gating flag marks them as a plaintext-key buffer rather than an integer + * key identifier. An address may occupy one word (FW-API, low 32 bits only) or + * two words (some base-API commands split it into low and high halves). + * + * When a field also names a size word, the buffer length carried there is + * validated too: the whole buffer [addr, addr + len) must fit inside the + * window, not just its start address. The check is written as len > end - addr + * (addr is already known to be < end) so it cannot overflow. + */ +int se_val_cmd_addrs(struct se_if_device_ctx *dev_ctx, struct se_api_msg *msg, + u32 tx_msg_sz, const struct se_cmd_addr_field *fields, + size_t count) +{ + const struct se_shared_mem *mem = &dev_ctx->se_shared_mem_mgmt.non_secure_mem; + u32 payload_words; + size_t i; + u64 base, end; + + if (!fields || !count) + return 0; + + if (!msg) + return -EINVAL; + + /* Number of complete u32 payload words present after the header. */ + if (tx_msg_sz < SE_MU_HDR_SZ) + return -EINVAL; + /* + * The caller-supplied byte count must agree with the size the firmware + * will act on (header word-size field, in 32-bit words), so a lying + * header cannot make us validate fewer words than are actually sent. + */ + if (tx_msg_sz != (u32)msg->header.size * sizeof(u32)) + return -EINVAL; + payload_words = (tx_msg_sz - SE_MU_HDR_SZ) / sizeof(u32); + + base = (u64)mem->dma_addr; + end = base + mem->size; + + /* A zero-sized or wrapping window can never contain a valid buffer. */ + if (end <= base) + return -EINVAL; + + for (i = 0; i < count; i++) { + const struct se_cmd_addr_field *f = &fields[i]; + u64 addr; + + /* Every word the field references must lie within the message. */ + if (f->lsb_idx >= payload_words) + return -EINVAL; + if (f->has_msb && f->msb_idx >= payload_words) + return -EINVAL; + + if (f->flag_idx != SE_CMD_ADDR_ALWAYS) { + bool flag_set; + + if (f->flag_idx >= payload_words) + return -EINVAL; + + flag_set = !!(msg->data[f->flag_idx] & f->flag_mask); + /* + * When the flag does not select DMA-address mode the + * word holds an integer key identifier; leave it alone. + */ + if (flag_set != f->is_addr_when_set) + continue; + } + + addr = msg->data[f->lsb_idx]; + if (f->has_msb) + addr |= (u64)msg->data[f->msb_idx] << 32; + + /* Zero marks an absent optional buffer. */ + if (!addr) + continue; + + if (addr < base || addr >= end) + return -EACCES; + + /* + * The whole buffer [addr, addr + len) must fit inside the + * window, not just its start. addr is already >= base and + * < end here, so end - addr is a positive value and every + * "len > end - addr" comparison below cannot overflow. Where + * the length comes from depends on f->size_idx. + */ + switch (f->size_idx) { + case SE_CMD_ADDR_DEDUCE_SZ: { + /* + * No length word is carried in the payload. Instead the + * buffer begins with a struct fw_tag_len_vers_info + * header whose length field gives the total buffer byte + * count. + */ + const struct fw_tag_len_vers_info *hdr; + u64 len; + + /* + * The header itself must lie inside the window before it + * can be read. addr is already >= base and < end, so + * end - addr is positive and cannot overflow. + */ + if (sizeof(*hdr) > end - addr) + return -EACCES; + + /* + * Translate the validated DMA address to its kernel + * virtual alias inside the coherent shared-memory + * mapping before dereferencing it; a raw DMA address + * must never be dereferenced directly. + */ + hdr = (const struct fw_tag_len_vers_info *) + (mem->ptr + (addr - base)); + len = le16_to_cpu(hdr->length); + if (!len || len > end - addr) + return -EACCES; + break; + } + case SE_CMD_ADDR_FIXED_SIZE: + /* buf_size: literal byte count (FW-defined constant). */ + if (!f->buf_size) + return -EINVAL; + if ((u64)f->buf_size > end - addr) + return -EACCES; + break; + case SE_CMD_RCVR_ADDR_VAR_SIZE: { + struct cmd_rcvr_data_info *crcvr_info = + &dev_ctx->priv->crcvr_info; + /* + * Export-response buffer: the size was supplied by FW + * in the preceding export command and stored per SE + * interface in cmd_rcvr_var_size. + */ + if ((u64)crcvr_info->cmd_rcvr_var_size > end - addr) + return -EACCES; + break; + } + default: { + /* size_idx names a payload word carrying the length. */ + u64 len; + + if (f->size_idx >= payload_words) + return -EINVAL; + + /* size_mask == 0 with a valid size_idx is a descriptor bug. */ + if (!f->size_mask) + return -EINVAL; + + /* + * Widen to u64 before shifting: size_shift is u8 and + * shifting a u32 by >= 32 is undefined behaviour. + */ + len = ((u64)msg->data[f->size_idx] >> f->size_shift) & f->size_mask; + if (len > end - addr) + return -EACCES; + break; + } + } + } + + return 0; +} /** * se_update_msg_chksum() - calculate and update message checksum word. @@ -45,6 +256,26 @@ int se_update_msg_chksum(u32 *msg, u32 msg_len) return 0; } +static void se_mark_fw_busy(struct se_if_device_ctx *dev_ctx) +{ + struct se_if_priv *priv = dev_ctx->priv; + struct fw_busy_info *fbusy_info = &priv->fw_busy_info; + unsigned long flags; + + spin_lock_irqsave(&fbusy_info->fw_busy_lock, flags); + if (!fbusy_info->fw_busy_dev_ctx) { + kref_get(&dev_ctx->refcount); + fbusy_info->fw_busy_dev_ctx = dev_ctx; + atomic_set(&fbusy_info->fw_busy, 1); + } + spin_unlock_irqrestore(&fbusy_info->fw_busy_lock, flags); +} + +void set_se_rcv_msg_timeout(struct se_if_device_ctx *dev_ctx, u32 timeout_ms) +{ + dev_ctx->rcv_msg_timeout_jiffies = msecs_to_jiffies(timeout_ms); +} + /** * ele_msg_rcv() - wait for a response from the secure enclave. * @dev_ctx: pointer to the SE dev context data. @@ -65,15 +296,26 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk struct se_if_priv *priv = dev_ctx->priv; bool is_rsp_wait_with_timeout = false; bool wait_uninterruptible = false; + bool wait_killable = false; unsigned long remaining_jiffies; unsigned long deadline_jiffies; unsigned long flags; int ret; - remaining_jiffies = msecs_to_jiffies(SE_RCV_MSG_DEFAULT_TIMEOUT_MS); + remaining_jiffies = dev_ctx->rcv_msg_timeout_jiffies; if (se_clbk_hdl == &priv->waiting_rsp_clbk_hdl) { is_rsp_wait_with_timeout = true; deadline_jiffies = jiffies + remaining_jiffies; + + /* + * Internal kernel transactions run on priv_dev_ctx (probe + * get_info/ping, FW auth, PM IMEM swap). They are not tied to a + * restartable syscall, so wait uninterruptibly: PM freezer fake + * signals must not abort them with -ERESTARTSYS. Userspace + * waiters stay interruptible via the deferred-signal path below. + */ + if (se_clbk_hdl->dev_ctx == priv->priv_dev_ctx) + wait_uninterruptible = true; } do { @@ -84,8 +326,14 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk /* Deadline hit: fence hung FW, like the ret==0 path. */ spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); se_clbk_hdl->rx_msg = NULL; - if (!completion_done(&se_clbk_hdl->done)) - atomic_set(&priv->fw_busy, 1); + /* rx_delivered is set only after a real response has + * been copied under clbk_rx_lock, so it correctly + * distinguishes a genuine timeout (no response → mark + * busy) from a spurious teardown-forced wakeup where + * the data is not yet safe to free. + */ + if (!se_clbk_hdl->rx_delivered) + se_mark_fw_busy(dev_ctx); spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); ret = -ETIMEDOUT; break; @@ -96,23 +344,57 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk if (wait_uninterruptible) ret = wait_for_completion_timeout(&se_clbk_hdl->done, remaining_jiffies); + else if (wait_killable) + ret = wait_for_completion_killable_timeout(&se_clbk_hdl->done, + remaining_jiffies); else ret = wait_for_completion_interruptible_timeout(&se_clbk_hdl->done, remaining_jiffies); if (ret == -ERESTARTSYS) { /* - * Record that a signal was observed, then continue waiting non- - * interruptibly until the response arrives or the timeout - * expires. The caller can surface the interruption to userspace - * after the protocol transaction is brought back to a - * synchronized state. + * First, non-fatal signal on the interruptible userspace + * path: defer it. Record that a signal was observed and keep + * waiting - now only killably - until the response arrives or + * the timeout expires. ele_msg_send_rcv() then surfaces the + * interruption to userspace as -ERESTARTSYS once the protocol + * transaction has resynchronised, so the in-flight command is + * neither abandoned nor re-sent. + * + * Waiting killably rather than fully uninterruptibly is what + * keeps a fatal signal (SIGKILL) able to terminate the task: + * a non-fatal signal no longer aborts the wait, but the task + * can never get stuck for the multi-thousand-second long + * timeout and trip the hung-task watchdog. */ - if (is_rsp_wait_with_timeout && + if (is_rsp_wait_with_timeout && !wait_killable && READ_ONCE(se_clbk_hdl->rx_msg)) { WRITE_ONCE(se_clbk_hdl->signal_rcvd, true); - wait_uninterruptible = true; + wait_killable = true; continue; } + + /* + * Command-receiver path, or a fatal signal on the + * killable path: the task is dying but the enclave may + * still DMA into the soon-to-be-freed buffer. Under + * clbk_rx_lock, drop rx_msg and arm fw_busy so a late + * callback cannot write freed memory. Exception: if + * rx_delivered is set a real response already landed, so + * report its size and keep the handle. + */ + if (is_rsp_wait_with_timeout) { + spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); + if (se_clbk_hdl->rx_delivered) { + ret = se_clbk_hdl->rx_msg_sz; + spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + break; + } + if (se_clbk_hdl->rx_msg) { + se_clbk_hdl->rx_msg = NULL; + se_mark_fw_busy(dev_ctx); + } + spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + } break; } @@ -132,8 +414,15 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); se_clbk_hdl->rx_msg = NULL; - if (!completion_done(&se_clbk_hdl->done)) - atomic_set(&priv->fw_busy, 1); + /* + * rx_delivered helps to decide if the circuit breaker is armed + * or not. rx_delivered is set only after a real response has + * been copied under clbk_rx_lock, so it correctly distinguishes + * a genuine timeout (no response → mark busy) from a spurious + * teardown-forced wakeup where the data is not yet safe to free. + */ + if (!se_clbk_hdl->rx_delivered) + se_mark_fw_busy(dev_ctx); spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); ret = -ETIMEDOUT; @@ -142,8 +431,35 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk get_se_if_name(priv->if_defs->se_if_type)); break; } + + /* + * A positive wait return normally means a real response. During + * teardown, se_if_probe_cleanup() forces this wait to return via + * complete_all() with no response, while the enclave may still + * DMA into the shared buffer. Treat that as a failed transaction + * and arm the circuit breaker so the buffer is quarantined, not + * freed. + * + * rx_delivered tells the two apart: se_if_rx_callback() sets it + * under clbk_rx_lock only after copying a real response. This + * keeps teardown-time session/storage close responses from being + * mistaken for the forced abort, which would fail the close and + * leak its DMA buffer. + */ + spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); + if (is_rsp_wait_with_timeout && atomic_read(&priv->going_away) && + !se_clbk_hdl->rx_delivered) { + se_clbk_hdl->rx_msg = NULL; + se_mark_fw_busy(dev_ctx); + spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + ret = -ENODEV; + break; + } + spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + ret = se_clbk_hdl->rx_msg_sz; break; + } while (ret < 0); return ret; @@ -196,13 +512,15 @@ int ele_msg_send(struct se_if_device_ctx *dev_ctx, return tx_msg_sz; } -static void ele_msg_send_rcv_cleanup(struct se_if_priv *priv) +static void ele_msg_send_rcv_cleanup(struct se_if_priv *priv, int *act_rx_msg_sz) { unsigned long flags; spin_lock_irqsave(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); priv->waiting_rsp_clbk_hdl.dev_ctx = NULL; priv->waiting_rsp_clbk_hdl.rx_msg = NULL; + if (act_rx_msg_sz) + *act_rx_msg_sz = priv->waiting_rsp_clbk_hdl.rx_msg_sz; priv->waiting_rsp_clbk_hdl.rx_msg_sz = 0; spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); } @@ -214,6 +532,10 @@ static void ele_msg_send_rcv_cleanup(struct se_if_priv *priv) * @tx_msg_sz: size of @tx_msg in bytes. * @rx_msg: caller-provided buffer to receive the response into. * @exp_rx_msg_sz: expected response size in bytes. + * @act_rx_msg_sz: optional output pointer; if non-NULL, receives the actual + * number of bytes copied into @rx_msg (min of FW-declared + * size and @exp_rx_msg_sz). Pass NULL when the caller does not + * need this value. * * Holds the SE command lock for the duration of the exchange to prevent * concurrent transactions. Signals are deferred until the protocol @@ -223,29 +545,63 @@ static void ele_msg_send_rcv_cleanup(struct se_if_priv *priv) * Return: number of bytes received on success, negative errno on error. */ int ele_msg_send_rcv(struct se_if_device_ctx *dev_ctx, void *tx_msg, - int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz) + int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz, int *act_rx_msg_sz) { struct se_if_priv *priv = dev_ctx->priv; + struct fw_busy_info *fbusy_info = &priv->fw_busy_info; + struct task_struct *msg_excl_owner; unsigned long flags; int err; guard(mutex)(&priv->se_if_cmd_lock); - if (atomic_read(&priv->fw_busy)) { - dev_dbg(priv->dev, "%s: ELE became unresponsive.\n", dev_ctx->devname); + /* + * Arm under clbk_rx_lock so the going_away check and arming are atomic + * against teardown (closes the lost-wakeup window); priv_dev_ctx close + * commands still pass. Check going_away before fw_busy so a caller + * racing unbind gets the permanent -ENODEV, not the retryable -EBUSY - + * these are deliberately distinct from the transient msg_if reservation. + */ + + spin_lock_irqsave(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); + msg_excl_owner = READ_ONCE(priv->msg_excl_flow.msg_excl_owner); + if (atomic_read(&priv->going_away) && msg_excl_owner != current) { + spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); + return -ENODEV; + } + /* + * fw_busy is the circuit breaker: while it is set, reject new + * transactions with -EBUSY. The one exception is a flow that has + * reserved this interface exclusively for itself via se_reserve_msg_if() + * by publishing its task in priv->msg_excl_flow.msg_excl_owner (e.g. + * the recovery flow in se_clear_fw_busy()). Only that owning task is let + * through here to issue its teardown-close messages; every other caller + * still gets -EBUSY. When the flow calls se_release_msg_if() the owner + * is cleared and the interface returns to general se_if_cmd_lock message + * exchange. The owner is only compared against current, so a stale read + * is harmless (a non-owner can never match) and no msg_excl_lock is + * needed here; there is no deadlock. + */ + if (atomic_read(&fbusy_info->fw_busy) && msg_excl_owner != current) { + spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); return -EBUSY; } + reinit_completion(&priv->waiting_rsp_clbk_hdl.done); - /* Publish rx_msg/rx_msg_sz under the lock read by se_if_rx_callback(). */ - spin_lock_irqsave(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); priv->waiting_rsp_clbk_hdl.dev_ctx = dev_ctx; priv->waiting_rsp_clbk_hdl.rx_msg_sz = exp_rx_msg_sz; priv->waiting_rsp_clbk_hdl.rx_msg = rx_msg; + /* + * Arm a fresh transaction: clear the delivered flag so a stale value + * from a previous response cannot make ele_msg_rcv() mistake a + * teardown-forced complete_all() for a genuine firmware response. + */ + priv->waiting_rsp_clbk_hdl.rx_delivered = false; spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); err = ele_msg_send(dev_ctx, tx_msg, tx_msg_sz); if (err < 0) { - ele_msg_send_rcv_cleanup(priv); + ele_msg_send_rcv_cleanup(priv, NULL); return err; } @@ -264,7 +620,7 @@ int ele_msg_send_rcv(struct se_if_device_ctx *dev_ctx, void *tx_msg, dev_ctx->devname, err); } - ele_msg_send_rcv_cleanup(priv); + ele_msg_send_rcv_cleanup(priv, act_rx_msg_sz); return err; } @@ -296,14 +652,15 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg) { struct se_clbk_handle *se_clbk_hdl; struct device *dev = mbox_cl->dev; + struct fw_busy_info *fbusy_info; + struct se_msg_hdr *header; + bool sz_mismatch = false; + struct se_if_priv *priv; /* * devname_snap: a local copy of dev_ctx->devname taken while * clbk_rx_lock is held. */ char devname_snap[32]; - struct se_msg_hdr *header; - bool sz_mismatch = false; - struct se_if_priv *priv; unsigned long flags; u32 rx_msg_sz; @@ -311,6 +668,8 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg) if (!priv) return; + fbusy_info = &priv->fw_busy_info; + /* The function can be called with NULL msg */ if (IS_ERR_OR_NULL(msg)) { dev_err(dev, "Message is invalid\n"); @@ -377,9 +736,37 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg) se_clbk_hdl = &priv->waiting_rsp_clbk_hdl; spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); if (!se_clbk_hdl->rx_msg) { - /* Close circuit breaker on spinlock race */ - atomic_set(&priv->fw_busy, 0); + /* + * Schedule fw_busy_work only while going_away is clear: + * teardown sets going_away under this lock before + * cancel_work_sync(), so scheduling after that would + * re-queue against freed priv (UAF). schedule_work() + * only enqueues (the handler runs later in process + * context, so se_clear_fw_busy()'s mutex is not taken + * under this spinlock). se_clear_fw_busy() also runs + * from teardown; both take fw_busy_lock first, so the + * second caller sees NULL and returns. + */ + if (atomic_read(&fbusy_info->fw_busy)) { + /* + * Snapshot the late response before scheduling + * fw_busy_work. se_clear_fw_busy() will parse + * this buffer to detect a session-open or + * storage-open response and immediately close + * the leaked firmware handle via + * fw_api_specific_ops(). The buffer is sized + * to MAX_ALLOWED_RX_MSG_SZ; clamp the copy + * length so an oversized FW message cannot + * overflow it. + */ + memcpy(fbusy_info->orphan_fw_rx_msg, msg, + min(rx_msg_sz, + (u32)MAX_ALLOWED_RX_MSG_SZ)); + if (!atomic_read(&priv->going_away)) + schedule_work(&fbusy_info->fw_busy_work); + } spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + dev_info(dev, "ELE responded (late), recovery FW available.\n"); return; } @@ -401,6 +788,12 @@ void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg) strscpy(devname_snap, se_clbk_hdl->dev_ctx->devname, sizeof(devname_snap)); memcpy(se_clbk_hdl->rx_msg, msg, se_clbk_hdl->rx_msg_sz); + /* + * Mark that a genuine firmware response was delivered. ele_msg_rcv() + * reads this under clbk_rx_lock to avoid mistaking this response for + * a teardown-forced complete_all() wakeup. + */ + se_clbk_hdl->rx_delivered = true; complete(&se_clbk_hdl->done); spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); diff --git a/drivers/firmware/imx/ele_common.h b/drivers/firmware/imx/ele_common.h index 89ae4c335e7b..332b35eb7db2 100644 --- a/drivers/firmware/imx/ele_common.h +++ b/drivers/firmware/imx/ele_common.h @@ -9,11 +9,15 @@ #include "se_ctrl.h" #define SE_RCV_MSG_DEFAULT_TIMEOUT_MS 3000 +#define SE_RCV_MSG_LONG_TIMEOUT_MS 5000000 #define ELE_SUCCESS_IND 0xD6 #define IMX_ELE_FW_DIR "imx/ele/" +#define MAX_WORD_SIZE 0x20 + +void set_se_rcv_msg_timeout(struct se_if_device_ctx *dev_ctx, u32 val); int se_update_msg_chksum(u32 *msg, u32 msg_len); int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk_hdl); @@ -21,13 +25,106 @@ int ele_msg_rcv(struct se_if_device_ctx *dev_ctx, struct se_clbk_handle *se_clbk int ele_msg_send(struct se_if_device_ctx *dev_ctx, void *tx_msg, int tx_msg_sz); int ele_msg_send_rcv(struct se_if_device_ctx *dev_ctx, void *tx_msg, - int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz); + int tx_msg_sz, void *rx_msg, int exp_rx_msg_sz, + int *act_rx_msg_sz); void se_if_rx_callback(struct mbox_client *mbox_cl, void *msg); int se_val_rsp_hdr_n_status(struct se_if_device_ctx *dev_ctx, struct se_api_msg *msg, u8 msg_id, u8 sz, u8 version); +/* + * Header that prefixes the input buffer of commands whose descriptor uses + * SE_CMD_ADDR_DEDUCE_SZ (e.g. ELE_OEM_AUTH_CONTAINER_REQ, + * ELE_KEYSTORE_REPROV_ENABLE_REQ). The command payload carries only the start + * address of that buffer, but its first four bytes follow this fixed layout: + * the 16-bit length field gives the total byte count of the buffer, so + * se_val_cmd_addrs() can bound-check the whole buffer, not just its start. The + * length is little-endian on the wire; read it with le16_to_cpu(). + */ +struct fw_tag_len_vers_info { + u8 version; + __le16 length; + u8 tag; +} __packed; + +/** + * struct se_cmd_addr_field - One DMA address embedded in an ELE command + * message payload. + * + * A number of ELE commands carry DMA physical addresses inside their message + * payload. se_val_cmd_addrs() range-checks each such address against the + * calling context's shared-memory window before the message reaches firmware. + * + * data[] index = message WORD index - 1, because the 4-byte se_msg_hdr is + * message WORD 0 and se_api_msg.data[0] is message WORD 1. + * + * @lsb_idx: data[] index of the low 32 bits of the address. + * @msb_idx: data[] index of the high 32 bits; valid only when @has_msb is + * set. Some base-API commands split the address into two words; + * FW-API commands do not. + * @has_msb: true when the address occupies two words (@lsb_idx + @msb_idx). + * @flag_idx: data[] index of the flag word that selects whether + * data[@lsb_idx] is a DMA address or an integer key identifier; + * %SE_CMD_ADDR_ALWAYS when the word is always a DMA address. + * @flag_mask: the selecting flag bit, already shifted to its position inside + * the 32-bit little-endian flag word. + * @is_addr_when_set: true when the word is a DMA address if the flag bit is + * set; false when it is an address if the bit is clear + * (inverse polarity, e.g. VERIFY_SIGN OPAQUE_KEY). + * @size_idx: data[] index of the word carrying the length in bytes of the + * buffer at this address, or one of the size-source sentinels + * below. se_val_cmd_addrs() uses the resulting length to confirm + * the whole buffer [addr, addr + len) fits inside the + * shared-memory window, not just its start. + * %SE_CMD_ADDR_FIXED_SIZE when the length is a firmware-defined + * literal supplied in @buf_size. + * %SE_CMD_RCVR_ADDR_VAR_SIZE when the length is taken from + * se_if_priv.cmd_rcvr_var_size (command-receiver responses). + * %SE_CMD_ADDR_DEDUCE_SZ when no length word is carried in the + * payload, but the buffer starts with a struct fw_tag_len_vers_info + * header whose length field gives the total buffer byte count; + * se_val_cmd_addrs() reads that header to range-check the whole + * buffer. + * @size_shift: right shift applied to the size word before masking, for a + * length packed into the high half of a word. + * @size_mask: bitmask applied after @size_shift to extract the length from + * the message word (0xFFFFFFFF for a full 32-bit length, 0xFFFF + * for a u16, 0xFF for a u8). Used only when @size_idx names a + * payload word; zero for the sentinel values. + * @buf_size: firmware-defined literal byte count. Used only when + * @size_idx == %SE_CMD_ADDR_FIXED_SIZE, where the whole buffer + * [addr, addr + @buf_size) must fit inside the shared-memory + * window. Left zero for every other @size_idx value. + */ +struct se_cmd_addr_field { + u8 lsb_idx; + u8 msb_idx; + bool has_msb; + u8 flag_idx; + u32 flag_mask; + bool is_addr_when_set; + u8 size_idx; + u8 size_shift; + u32 size_mask; + u32 buf_size; +}; + +#define SE_CMD_ADDR_ALWAYS 0xEFu +/* size is the literal in buf_size */ +#define SE_CMD_ADDR_FIXED_SIZE 0xFDu +/* size from se_if_priv.cmd_rcvr_var_size */ +#define SE_CMD_RCVR_ADDR_VAR_SIZE 0xFEu +/* size read from the buffer's fw_tag_len_vers_info header */ +#define SE_CMD_ADDR_DEDUCE_SZ 0xFFu + +int se_val_cmd_addrs(struct se_if_device_ctx *dev_ctx, struct se_api_msg *msg, + u32 tx_msg_sz, const struct se_cmd_addr_field *fields, + size_t count); + +const struct se_cmd_addr_field *ele_fw_cmd_addr_fields(u8 cmd, size_t *count); +const struct se_cmd_addr_field *ele_fw_rsp_addr_fields(u8 cmd, size_t *count); +const struct se_cmd_addr_field *ele_base_cmd_addr_fields(u8 cmd, size_t *count); /* Fill a command message header with a given command ID and length in bytes. */ static inline void se_fill_cmd_msg_hdr(struct se_if_priv *priv, struct se_msg_hdr *hdr, u8 cmd, u32 len, bool is_base_api) @@ -42,4 +139,9 @@ int se_save_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem); int se_restore_imem_state(struct se_if_priv *priv, struct se_imem_buf *imem); +int se_chk_tx_rsp_msg_hdr(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz); +int se_chk_tx_cmd_msg_hdr(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz, u32 rx_msg_sz); + #endif /*__ELE_COMMON_H__ */ diff --git a/drivers/firmware/imx/ele_fw_api.c b/drivers/firmware/imx/ele_fw_api.c new file mode 100644 index 000000000000..42cda7de66e2 --- /dev/null +++ b/drivers/firmware/imx/ele_fw_api.c @@ -0,0 +1,404 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2026 NXP + */ + +#include "se_ctrl.h" +#include "ele_common.h" +#include "ele_fw_api.h" + +static int se_cmd_receiver_allowed_cmd(struct se_if_device_ctx *dev_ctx, + struct se_api_msg *msg, u32 tx_msg_sz) +{ + u8 cmd = msg->header.command; + + switch (cmd) { + case ELE_SESSION_CLOSE_REQ: + if (tx_msg_sz < ELE_SESSION_CLOSE_REQ_SZ || + msg->data[0] != dev_ctx->sess_hdl) + return -EINVAL; + return 0; + case ELE_STORAGE_CLOSE_REQ: + if (tx_msg_sz < ELE_STORAGE_CLOSE_REQ_SZ || + msg->data[0] != dev_ctx->strg_hdl) + return -EINVAL; + return 0; + case ELE_STORAGE_MASTER_IMPORT_REQ: { + const struct se_cmd_addr_field *fields; + size_t count; + + fields = ele_fw_cmd_addr_fields(cmd, &count); + return se_val_cmd_addrs(dev_ctx, msg, tx_msg_sz, fields, count); + } + default: + return -EOPNOTSUPP; + } +} + +static int se_cmd_receiver_allowed_rsp(struct se_if_device_ctx *dev_ctx, + struct se_api_msg *msg, u32 tx_msg_sz) +{ + struct cmd_rcvr_data_info *crcvr_info = &dev_ctx->priv->crcvr_info; + const struct se_cmd_addr_field *fields; + u8 cmd = msg->header.command; + size_t count; + + switch (cmd) { + case ELE_STORAGE_EXPORT_FINISH_REQ: + case ELE_STORAGE_CHUNK_GET_DONE_REQ: + case ELE_STORAGE_CHUNK_DELETE_REQ: + return 0; + default: + /* + * These responses supply a kernel buffer address to firmware. + * Range-check the embedded DMA address against the calling + * context's shared-memory window before the message is sent. + */ + if (crcvr_info->cmd_rcvr_last_rcvd_cmd_id != cmd) + return -EINVAL; + + fields = ele_fw_rsp_addr_fields(cmd, &count); + if (!count) + return -EOPNOTSUPP; + + return se_val_cmd_addrs(dev_ctx, msg, tx_msg_sz, fields, count); + } +} + +int ele_uapi_allowed_fw_rsp(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz) +{ + struct se_api_msg *msg = container_of(header, struct se_api_msg, header); + struct se_if_priv *priv = dev_ctx->priv; + + scoped_guard(mutex, &priv->modify_lock) + if (dev_ctx != priv->cmd_receiver_clbk_hdl.dev_ctx) + return -EINVAL; + + return se_cmd_receiver_allowed_rsp(dev_ctx, msg, tx_msg_sz); +} + +int ele_uapi_allowed_fw_cmd(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz, u32 rx_msg_sz) +{ + struct se_api_msg *msg = container_of(header, struct se_api_msg, header); + struct se_if_priv *priv = dev_ctx->priv; + const struct se_cmd_addr_field *fields; + bool receiver_exists = false; + bool is_cmd_receiver = false; + size_t count; + int ret = 0; + + scoped_guard(mutex, &priv->modify_lock) { + if (priv->cmd_receiver_clbk_hdl.dev_ctx) + receiver_exists = true; + if (dev_ctx == priv->cmd_receiver_clbk_hdl.dev_ctx) + is_cmd_receiver = true; + } + + if (is_cmd_receiver && header->tag == priv->if_defs->cmd_tag) + return se_cmd_receiver_allowed_cmd(dev_ctx, msg, tx_msg_sz); + + /* Reject any response message with non-command receiver */ + if (header->tag == priv->if_defs->rsp_tag) + return -EOPNOTSUPP; + + /* Reject any other tag */ + if (header->tag != priv->if_defs->cmd_tag) + return -EOPNOTSUPP; + + /* + * Identify the command first. Session/storage commands enforce their + * own-handle checks; crypto commands that embed DMA addresses defer to + * the shared range check below. Any command not named here is left with + * ret == 0 (permitted) as before. + */ + switch (header->command) { + case ELE_SESSION_OPEN_REQ: + /* Might be cleared as part of tear down. */ + ret = dev_ctx->sess_hdl ? -EEXIST : 0; + if (rx_msg_sz < ELE_SESSION_OPEN_RSP_SZ) + ret = -EINVAL; + break; + case ELE_SESSION_CLOSE_REQ: + /* Might be cleared as part of tear down. */ + if (!dev_ctx->sess_hdl) { + ret = -ENXIO; + break; + } + /* + * A close request must target this context's own session. The + * handle to close is carried in the payload (data[0]); reject a + * request whose buffer is too short to hold it, or whose handle + * does not match this context. Checking the buffer size first + * also keeps the data[0] read in bounds. This stops one process + * from closing - and leaking - another process's session with a + * spoofed handle. + */ + if (tx_msg_sz < ELE_SESSION_CLOSE_REQ_SZ || + msg->data[0] != dev_ctx->sess_hdl) + ret = -EINVAL; + break; + case ELE_FW_GET_INFO_REQ: + case ELE_KEY_STORE_OPEN_REQ: + case ELE_KEY_STORE_CLOSE_REQ: + case ELE_KEY_MGMT_OPEN_REQ: + case ELE_KEY_MGMT_CLOSE_REQ: + case ELE_MANAGE_KEY_GROUP_REQ: + case ELE_GET_KEY_ATTR_REQ: + case ELE_KEY_DELETE_REQ: + case ELE_MAC_OPEN_REQ: + case ELE_MAC_CLOSE_REQ: + case ELE_CIPHER_OPEN_REQ: + case ELE_CIPHER_CLOSE_REQ: + case ELE_SIGNATURE_GENERATE_OPEN_REQ: + case ELE_SIGNATURE_GENERATE_CLOSE_REQ: + case ELE_SIGNATURE_VERIFY_OPEN_REQ: + case ELE_SIGNATURE_VERIFY_CLOSE_REQ: + case ELE_DATA_STORAGE_OPEN_REQ: + case ELE_DATA_STORAGE_CLOSE_REQ: + case ELE_DATA_DELETE_REQ: + ret = 0; + break; + case ELE_STORAGE_OPEN_REQ: + /* Might be cleared as part of tear down. */ + if (dev_ctx->strg_hdl) { + ret = -EEXIST; + break; + } + /* + * NOTE: this advisory early check is intentionally not the + * definitive exclusivity gate. modify_lock is dropped before the + * command is sent, creating a TOCTOU window. That window is + * closed by two additional layers: + * 1. se_if_cmd_lock, held for the full send+receive cycle, so + * only one ELE_STORAGE_OPEN_REQ is in flight per MU at a + * time. + * 2. set_dev_ctx_as_command_receiver(), which re-checks under + * modify_lock after the response arrives. If two callers + * race past this check, FW itself rejects the second + * ELE_STORAGE_OPEN_REQ before any handle is allocated. + * See Documentation/driver-api/firmware/other_interfaces.rst, + * section "ELE_STORAGE_OPEN_REQ concurrency and + * command-receiver exclusivity". + */ + if (receiver_exists && !is_cmd_receiver) + ret = -EBUSY; + if (rx_msg_sz < ELE_STORAGE_OPEN_RSP_SZ) + ret = -EINVAL; + break; + case ELE_STORAGE_CLOSE_REQ: + /* Might be cleared as part of tear down. */ + if (!dev_ctx->strg_hdl) { + ret = -ENXIO; + break; + } + /* Same self-ownership check as the session close above. */ + if (tx_msg_sz < ELE_STORAGE_CLOSE_REQ_SZ || + msg->data[0] != dev_ctx->strg_hdl) + ret = -EINVAL; + break; + case ELE_STORAGE_STATUS_REQ: + ret = 0; + break; + default: + /* FW commands that embed DMA addresses. */ + fields = ele_fw_cmd_addr_fields(header->command, &count); + if (!count) { + ret = -EOPNOTSUPP; + break; + } + + ret = se_val_cmd_addrs(dev_ctx, msg, tx_msg_sz, fields, count); + break; + } + + return ret; +} + +void cmd_receiver_specific_ops(struct se_if_device_ctx *dev_ctx, + struct se_api_msg *rx_msg) +{ + struct cmd_rcvr_data_info *crcvr_info = &dev_ctx->priv->crcvr_info; + struct se_msg_hdr *header = &rx_msg->header; + + crcvr_info->cmd_rcvr_last_rcvd_cmd_id = 0; + crcvr_info->cmd_rcvr_var_size = 0; + switch (header->command) { + case ELE_STORAGE_MASTER_EXPORT_REQ: + /* + * FW sent an export-start command with key_store_size at + * data[1]. Save it so se_val_cmd_addrs() can range-check the + * response buffer when the cmd_receiver sends back the address. + */ + crcvr_info->cmd_rcvr_last_rcvd_cmd_id = ELE_STORAGE_MASTER_EXPORT_REQ; + crcvr_info->cmd_rcvr_var_size = rx_msg->data[1]; + break; + case ELE_STORAGE_CHUNK_EXPORT_REQ: + /* + * FW sent a chunk-export command with chunk_size at data[1]. + * Save it so se_val_cmd_addrs() can range-check the response + * buffer when the cmd_receiver sends back the address. + */ + crcvr_info->cmd_rcvr_last_rcvd_cmd_id = ELE_STORAGE_CHUNK_EXPORT_REQ; + crcvr_info->cmd_rcvr_var_size = rx_msg->data[1]; + break; + case ELE_STORAGE_CHUNK_GET_REQ: + crcvr_info->cmd_rcvr_last_rcvd_cmd_id = ELE_STORAGE_CHUNK_GET_REQ; + break; + } +} + +int fw_api_specific_ops(struct se_if_device_ctx *dev_ctx, struct se_api_msg *rx_msg, + bool is_cmd_interrupted) +{ + struct se_msg_hdr *header = &rx_msg->header; + struct se_if_priv *priv = dev_ctx->priv; + + switch (header->command) { + case ELE_SESSION_OPEN_REQ: + dev_ctx->sess_hdl = rx_msg->data[1]; + if (is_cmd_interrupted) { + if (se_close_session(dev_ctx, dev_ctx->sess_hdl)) + dev_err(dev_ctx->priv->dev, "failed to close session.\n"); + dev_ctx->sess_hdl = 0; + } + break; + case ELE_SESSION_CLOSE_REQ: + dev_ctx->sess_hdl = 0; + break; + case ELE_STORAGE_OPEN_REQ: { + int rc; + + /* + * Record the storage handle before registering as command + * receiver. FW has already allocated the handle; if we assigned + * it only after a successful registration, a failing + * set_dev_ctx_as_command_receiver() (e.g. -EBUSY) would leave + * strg_hdl at 0 while the ioctl still returns success to + * userspace. The kernel would then never close the handle on + * teardown, leaking it in FW. Storing it first guarantees + * cleanup_dev_ctx() closes it on the next close(), regardless + * of whether registration succeeded. + */ + dev_ctx->strg_hdl = rx_msg->data[1]; + + rc = is_cmd_interrupted ? 0 : set_dev_ctx_as_command_receiver(dev_ctx); + if (is_cmd_interrupted || rc) { + if (se_close_storage(dev_ctx, dev_ctx->strg_hdl)) + dev_err(dev_ctx->priv->dev, "failed to close storage.\n"); + dev_ctx->strg_hdl = 0; + if (rc) + dev_err(priv->dev, + "Failed to register %s as CMD-Receiver: %d\n", + dev_ctx->devname, rc); + return rc; + } + break; + } + case ELE_STORAGE_CLOSE_REQ: + scoped_guard(mutex, &priv->modify_lock) + unset_dev_ctx_as_command_receiver(dev_ctx); + dev_ctx->strg_hdl = 0; + break; + } + + return 0; +} + +int se_close_session(struct se_if_device_ctx *dev_ctx, u32 session_hdl) +{ + struct se_api_msg *tx_msg __free(kfree) = + kzalloc(ELE_SESSION_CLOSE_REQ_SZ, GFP_KERNEL); + struct se_api_msg *rx_msg __free(kfree) = + kzalloc(ELE_SESSION_CLOSE_RSP_SZ, GFP_KERNEL); + struct se_if_priv *priv; + int ret; + + if (!dev_ctx || !dev_ctx->priv) + return -EINVAL; + + if (!tx_msg || !rx_msg) + return -ENOMEM; + + priv = dev_ctx->priv; + + /* + * Session close is a FW-API command; pass is_base_api=false so the + * header carries fw_api_ver. + */ + se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header, + ELE_SESSION_CLOSE_REQ, ELE_SESSION_CLOSE_REQ_SZ, false); + + tx_msg->data[0] = session_hdl; + + /* + * Transmit on the caller's own context. Using dev_ctx (rather than + * hardcoding priv->priv_dev_ctx) keeps a userspace close() subject to + * the going_away check in ele_msg_send_rcv(): if unbind has begun and + * freed priv->tx_chan, the send is rejected with -ENODEV instead of + * touching the freed mailbox channel. + * + * The teardown and fw_busy-recovery paths instead reserve the messaging + * interface for their own task via se_reserve_msg_if() before calling + * this. That reservation (msg_excl_owner == current) is what makes + * ele_msg_send_rcv() let their resync closes through the going_away and + * fw_busy gates; they also pass priv_dev_ctx so the wait is the + * uninterruptible internal-context wait. + */ + + ret = ele_msg_send_rcv(dev_ctx, + tx_msg, + ELE_SESSION_CLOSE_REQ_SZ, + rx_msg, + ELE_SESSION_CLOSE_RSP_SZ, NULL); + if (ret < 0) + return ret; + + ret = se_val_rsp_hdr_n_status(dev_ctx, + rx_msg, + ELE_SESSION_CLOSE_REQ, + ELE_SESSION_CLOSE_RSP_SZ, + priv->if_defs->fw_api_ver); + return ret; +} + +int se_close_storage(struct se_if_device_ctx *dev_ctx, u32 storage_hdl) +{ + struct se_api_msg *tx_msg __free(kfree) = + kzalloc(ELE_STORAGE_CLOSE_REQ_SZ, GFP_KERNEL); + struct se_api_msg *rx_msg __free(kfree) = + kzalloc(ELE_STORAGE_CLOSE_RSP_SZ, GFP_KERNEL); + struct se_if_priv *priv; + int ret; + + if (!dev_ctx || !dev_ctx->priv) + return -EINVAL; + + if (!tx_msg || !rx_msg) + return -ENOMEM; + + priv = dev_ctx->priv; + + /* Same FW-API version handling as se_close_session() above. */ + se_fill_cmd_msg_hdr(priv, (struct se_msg_hdr *)&tx_msg->header, + ELE_STORAGE_CLOSE_REQ, ELE_STORAGE_CLOSE_REQ_SZ, false); + + tx_msg->data[0] = storage_hdl; + + /* Transmit on the caller's own context; see se_close_session(). */ + ret = ele_msg_send_rcv(dev_ctx, + tx_msg, + ELE_STORAGE_CLOSE_REQ_SZ, + rx_msg, + ELE_STORAGE_CLOSE_RSP_SZ, NULL); + if (ret < 0) + return ret; + + ret = se_val_rsp_hdr_n_status(dev_ctx, + rx_msg, + ELE_STORAGE_CLOSE_REQ, + ELE_STORAGE_CLOSE_RSP_SZ, + priv->if_defs->fw_api_ver); + return ret; +} diff --git a/drivers/firmware/imx/ele_fw_api.h b/drivers/firmware/imx/ele_fw_api.h new file mode 100644 index 000000000000..7a6d7dab84ff --- /dev/null +++ b/drivers/firmware/imx/ele_fw_api.h @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright 2026 NXP + */ + +#ifndef ELE_FW_API_H +#define ELE_FW_API_H +#include "se_ctrl.h" + +#define ELE_SESSION_OPEN_REQ 0x10u +#define ELE_SESSION_OPEN_RSP_SZ 0x0Cu + +#define ELE_SESSION_CLOSE_REQ_SZ 0x08u +#define ELE_SESSION_CLOSE_RSP_SZ 0x08u +#define ELE_SESSION_CLOSE_REQ 0x11u + +/* + * Session-scoped FW-API service, key-management and close command opcodes + * (SAB command IDs, PSA_COMPLIANT message layout). These commands do not + * embed DMA staging-buffer addresses that require range-checking; they are + * defined here for completeness and for use by the command allow-list. + * ELE_FW_GET_INFO_REQ is the FW-API get-info opcode and is intentionally + * distinct from the base-API ELE_GET_INFO_REQ (0xda) in ele_base_msg.h. + */ +#define ELE_FW_GET_INFO_REQ 0x16u +#define ELE_KEY_STORE_OPEN_REQ 0x30u +#define ELE_KEY_STORE_CLOSE_REQ 0x31u +#define ELE_KEY_MGMT_OPEN_REQ 0x40u +#define ELE_KEY_MGMT_CLOSE_REQ 0x41u +#define ELE_MANAGE_KEY_GROUP_REQ 0x45u +#define ELE_GET_KEY_ATTR_REQ 0x4Cu +#define ELE_KEY_DELETE_REQ 0x4Eu +#define ELE_MAC_OPEN_REQ 0x50u +#define ELE_MAC_CLOSE_REQ 0x51u +#define ELE_CIPHER_OPEN_REQ 0x60u +#define ELE_CIPHER_CLOSE_REQ 0x61u +#define ELE_SIGNATURE_GENERATE_OPEN_REQ 0x70u +#define ELE_SIGNATURE_GENERATE_CLOSE_REQ 0x71u +#define ELE_SIGNATURE_VERIFY_OPEN_REQ 0x80u +#define ELE_SIGNATURE_VERIFY_CLOSE_REQ 0x81u +#define ELE_DATA_STORAGE_OPEN_REQ 0xA0u +#define ELE_DATA_STORAGE_CLOSE_REQ 0xA1u +#define ELE_DATA_DELETE_REQ 0xA4u + +/* + * FW-API crypto command opcodes that embed one or more DMA physical addresses + * in their message payload. ele_uapi_allowed_fw_cmd() range-checks those + * addresses against the calling context's shared-memory window before the + * message is handed to firmware. Opcodes match the SAB command IDs emitted by + * the userspace library (PSA_COMPLIANT message layout). + */ +#define ELE_PUB_KEY_EXPORT_REQ 0x32u +#define ELE_KEYSTORE_REPROV_ENABLE_REQ 0x3Fu +#define ELE_KEYGEN_REQ 0x42u +#define ELE_KEY_EXCHANGE_REQ 0x47u +#define ELE_KEY_IMPORT_REQ 0x4Fu +#define ELE_KEY_IMPORT 0x4Fu +#define ELE_MAC_REQ 0x52u +#define ELE_CIPHER_REQ 0x62u +#define ELE_AUTH_ENC_REQ 0x64u +#define ELE_AUTH_ENC_NEW_REQ 0x65u +#define ELE_SIGNATURE_GENERATE_REQ 0x72u +#define ELE_PUB_KEY_ATTEST_REQ 0x74u +#define ELE_SIGNATURE_VERIFY_REQ 0x82u +#define ELE_DATA_STORAGE_REQ 0xA2u +#define ELE_ENC_DATA_STORAGE_REQ 0xA3u +#define ELE_ASYMMETRIC_ENC_REQ 0x92u + +#define ELE_KEY_GENERIC_CRYPTO_REQ 0xC2u +#define ELE_GC_CIPHER_REQ 0xC8u +#define ELE_GC_AEAD_REQ 0xC9u +#define ELE_GC_ACRYPTO_REQ 0xCAu +#define ELE_GC_AKEY_GEN_REQ 0xCBu +#define ELE_HASH_ONE_GO_REQ 0xCCu +#define ELE_RNG_GET_RANDOM_REQ 0xCDu + +#define ELE_STORAGE_OPEN_REQ 0xE0u +#define ELE_STORAGE_OPEN_RSP_SZ 0x0Cu + +#define ELE_STORAGE_CLOSE_REQ_SZ 0x08u +#define ELE_STORAGE_CLOSE_RSP_SZ 0x08u +#define ELE_STORAGE_CLOSE_REQ 0xE1u + +#define ELE_STORAGE_MASTER_IMPORT_REQ 0xE2u +#define ELE_STORAGE_MASTER_EXPORT_REQ 0xE3u +#define ELE_STORAGE_EXPORT_FINISH_REQ 0xE4u +#define ELE_STORAGE_CHUNK_EXPORT_REQ 0xE5u +#define ELE_STORAGE_CHUNK_GET_REQ 0xE6u +#define ELE_STORAGE_CHUNK_GET_DONE_REQ 0xE7u +#define ELE_STORAGE_CHUNK_DELETE_REQ 0xE9u +#define ELE_STORAGE_STATUS_REQ 0xEAu + +int ele_uapi_allowed_fw_rsp(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz); +int ele_uapi_allowed_fw_cmd(struct se_if_device_ctx *dev_ctx, struct se_msg_hdr *header, + u32 tx_msg_sz, u32 rx_msg_sz); +int fw_api_specific_ops(struct se_if_device_ctx *dev_ctx, struct se_api_msg *rx_msg, + bool is_cmd_interrupted); +void cmd_receiver_specific_ops(struct se_if_device_ctx *dev_ctx, + struct se_api_msg *rx_msg); +int se_close_session(struct se_if_device_ctx *dev_ctx, u32 session_hdl); +int se_close_storage(struct se_if_device_ctx *dev_ctx, u32 storage_hdl); + +#endif /* ELE_FW_API_H */ diff --git a/drivers/firmware/imx/ele_msg_addr_field.c b/drivers/firmware/imx/ele_msg_addr_field.c new file mode 100644 index 000000000000..ac49ac78da68 --- /dev/null +++ b/drivers/firmware/imx/ele_msg_addr_field.c @@ -0,0 +1,619 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright 2026 NXP + */ + +#include + +#include "ele_common.h" +#include "ele_base_msg.h" +#include "ele_fw_api.h" + +/* + * Base-API commands that embed one or more DMA physical addresses in their + * payload. Unlike the FW-API crypto commands, GET_INFO and DEV_ATTEST split + * their response-buffer address across two words: the high half is written + * first (lower word index) and the low half next, so has_msb is set and the + * msb_idx precedes the lsb_idx. GEN_KEY_BLOB uses single-word LSB addresses. + * See struct se_cmd_addr_field in ele_common.h for the field semantics. + */ +static const struct se_cmd_addr_field ele_get_info_addr_fields[] = { + /* + * rsp_data_addr_hi @ data[0], rsp_data_addr_lo @ data[1]; + * buf_sz is a u16 in the low half of data[2]. + */ + { .lsb_idx = 1, .msb_idx = 0, .has_msb = true, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFu }, +}; + +static const struct se_cmd_addr_field ele_dev_attest_addr_fields[] = { + /* + * rsp_data_addr_hi @ data[0], rsp_data_addr_lo @ data[1]; + * buf_sz is a u16 in the low half of data[2]. + */ + { .lsb_idx = 1, .msb_idx = 0, .has_msb = true, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFu }, +}; + +static const struct se_cmd_addr_field ele_oem_auth_cntr_addr_fields[] = { + /* + * Container Header address: a 64-bit physical address split across two + * words. data[0] holds the 32-bit MSB and data[1] holds the 32-bit LSB + * (ELE API spec Table 27, word size = 0x3, so the command is header + + * MSB + LSB only). No length word is carried in the payload; the buffer + * length is read from the fw_tag_len_vers_info header at its start + * (SE_CMD_ADDR_DEDUCE_SZ) so the whole buffer is range-checked. + */ + { .lsb_idx = 1, .msb_idx = 0, .has_msb = true, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = SE_CMD_ADDR_DEDUCE_SZ }, /* container_hdr_addr */ +}; + +/* + * GENERATE ELE KEY BLOB command (ELE_GEN_KEY_BLOB_REQ, 0xAF). + * ELE API spec Table 73, word size = 0x8 (header + 7 data words): + * data[0] = key_identifier + * data[1] = Reserved + * data[2] = load_address (32-bit; must be 64-bit aligned) + * data[3] = Reserved + * data[4] = store_address (32-bit; must be 64-bit aligned) + * data[5] = Reserved[31:16] | max_export_size[15:0] + * data[6] = CRC + * + * load_addr points to the input: a blob header (8 bytes, Table 77) followed + * by the plaintext payload. No size word is present in the message for this + * input buffer. The maximum input size is determined by the largest supported + * payload type: OTFAD key configuration (0x28 bytes per Table 79) plus the + * 8-byte header gives 0x30 bytes. That is the literal upper bound placed in + * buf_size and selected with SE_CMD_ADDR_FIXED_SIZE, so se_val_cmd_addrs() + * can verify [load_addr, load_addr+0x30) lies within the shared-memory + * window. + */ +#define OP_GEN_ELE_KEY_BLOB_INPUT_MAX_SZ 0x30 /* blob hdr (8) + OTFAD payload (0x28) */ +static const struct se_cmd_addr_field ele_gen_key_blob_addr_fields[] = { + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = SE_CMD_ADDR_FIXED_SIZE, + .buf_size = OP_GEN_ELE_KEY_BLOB_INPUT_MAX_SZ }, /* load_address */ + /* store_address @ data[4]; max_export_size is u16 in low half of data[5] */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFu }, /* store_address */ +}; + +/* + * Return the address-field descriptor table for a base-API command, or NULL + * when the command embeds no DMA addresses. count is set to the number of + * entries. + */ +const struct se_cmd_addr_field *ele_base_cmd_addr_fields(u8 cmd, size_t *count) +{ + switch (cmd) { + case ELE_OEM_AUTH_CONTAINER_REQ: + *count = ARRAY_SIZE(ele_oem_auth_cntr_addr_fields); + return ele_oem_auth_cntr_addr_fields; + case ELE_GEN_KEY_BLOB_REQ: + *count = ARRAY_SIZE(ele_gen_key_blob_addr_fields); + return ele_gen_key_blob_addr_fields; + case ELE_GET_INFO_REQ: + *count = ARRAY_SIZE(ele_get_info_addr_fields); + return ele_get_info_addr_fields; + case ELE_DEV_ATTEST_REQ: + *count = ARRAY_SIZE(ele_dev_attest_addr_fields); + return ele_dev_attest_addr_fields; + default: + *count = 0; + return NULL; + } +} + +/* + * FW-API crypto commands that embed one or more DMA physical addresses in + * their payload. On the PSA_COMPLIANT ABI most addresses are written by the + * userspace library as a single little-endian 32-bit LSB word (the high half + * is always zero), so has_msb is left false for those entries. A few commands + * (pub-key-export 0x32, keystore reprov-enable 0x3F) carry an explicit ext/MSB + * word ahead of the LSB word, matching the base-API two-word address layout; + * their entries set has_msb = true so the MSB word is validated too. See + * struct se_cmd_addr_field in ele_common.h for the field semantics. + */ +static const struct se_cmd_addr_field ele_pub_key_export_addr_fields[] = { + /* + * out_key_addr: the recovered public key output buffer. Its high half + * out_key_addr_ext is data[2] and its low half out_key_addr is data[3]; + * the library always writes it via set_phy_addr_to_words(), so it is + * always a DMA address. Its length is out_key_size, the u16 in the low + * half of data[4]. key_identifier (data[1]) is an integer, not an + * address. + */ + { .lsb_idx = 3, .msb_idx = 2, .has_msb = true, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFu }, /* out_key_addr */ +}; + +static const struct se_cmd_addr_field ele_keystore_reprov_en_addr_fields[] = { + /* + * Signed message address: a 64-bit physical address split across two + * words. data[0] holds the 32-bit MSB and data[1] holds the 32-bit LSB + * (ELE API spec Table 202, word size = 0x3, so the command is header + + * MSB + LSB only). No length word is carried in the payload; the buffer + * length is read from the fw_tag_len_vers_info header at its start + * (SE_CMD_ADDR_DEDUCE_SZ) so the whole buffer is range-checked. + */ + { .lsb_idx = 1, .msb_idx = 0, .has_msb = true, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = SE_CMD_ADDR_DEDUCE_SZ }, /* signed_msg_addr */ +}; + +static const struct se_cmd_addr_field ele_keygen_addr_fields[] = { + /* + * key @ data[1]: a plaintext private-key output buffer only when the + * KEY_GENERATION PLAINTEXT_KEY flag (bit 3 of the flags byte in the low + * 8 bits of data[8]) is set; otherwise it is an integer key identifier. + * Its length is priv_key_sz, the u16 in the high half of data[8]. + */ + { .lsb_idx = 1, .flag_idx = 8, .flag_mask = 0x00000008u, .is_addr_when_set = true, + .size_idx = 8, .size_shift = 16, .size_mask = 0xFFFFu }, /* priv_key_addr */ + /* + * pub_key_addr @ data[9]: always a DMA address. Its length is + * pub_key_sz, the u16 in the low half of data[2]. + */ + { .lsb_idx = 9, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFu }, /* pub_key_addr */ +}; + +static const struct se_cmd_addr_field ele_key_exchange_addr_fields[] = { + /* + * PSA_COMPLIANT key-exchange payload. key_management_handle is data[0] + * and flags/reserved is data[1]; the four buffer addresses that follow + * are each written unconditionally via set_phy_addr_to_words() (single + * LSB word, high half always zero), so all are always DMA addresses. + * Each address is immediately followed by its full u32 byte length. + */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_mask = 0xFFFFFFFFu }, /* in_content_addr */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFFFFFu }, /* in_pub_buffer_addr */ + { .lsb_idx = 6, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 7, .size_mask = 0xFFFFFFFFu }, /* user_fixed_info_addr */ + { .lsb_idx = 8, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFFFFFu }, /* output_addr */ +}; + +static const struct se_cmd_addr_field ele_key_import_addr_fields[] = { + /* + * PSA_COMPLIANT key-import payload. key_management_handle is data[0] + * and flags/reserved is data[1]; the single input buffer address that + * follows is written unconditionally via set_phy_addr_to_words() + * (single LSB word, high half always zero), so it is always a DMA + * address. Its length is the full u32 input_size in data[3]. + */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_mask = 0xFFFFFFFFu }, /* input_address */ +}; + +/* ELE_MAC_REQ: MAC one-go operation. */ +static const struct se_cmd_addr_field ele_mac_addr_fields[] = { + /* key: plaintext-key buffer only when the MAC PLAINTEXT_KEY flag is set */ + { .lsb_idx = 1, .flag_idx = 5, .flag_mask = 0x00080000u, .is_addr_when_set = true, + .size_idx = 7, .size_mask = 0xFFFFu }, /* key_size */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* payload_address */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFu }, /* mac_address */ + { .lsb_idx = 8, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFu }, /* context_address */ +}; + +/* ELE_CIPHER_REQ: symmetric cipher one-go operation. */ +static const struct se_cmd_addr_field ele_cipher_addr_fields[] = { + /* key: plaintext-key buffer only when the CIPHER PLAINTEXT_KEY flag is set */ + { .lsb_idx = 1, .flag_idx = 3, .flag_mask = 0x00080000u, .is_addr_when_set = true, + .size_idx = 9, .size_mask = 0xFFFFu }, /* key_size */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_mask = 0xFFFFu }, /* iv_address */ + { .lsb_idx = 5, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 7, .size_mask = 0xFFFFFFFFu }, /* input_address */ + { .lsb_idx = 6, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 8, .size_mask = 0xFFFFFFFFu }, /* output_address */ + { .lsb_idx = 10, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 11, .size_mask = 0xFFFFu }, /* context_address */ +}; + +/* + * AEAD encrypt/decrypt legacy command (ELE_AUTH_ENC_REQ, 0x64). + * ELE API spec Table 287, word size = 0xD (header + 12 data words): + * data[0] = cipher_handle + * data[1] = key_identifier + * data[2] = IV LSB address + * data[3] = Reserved[31:24] | Flags[23:16] | IV_size[15:0] + * data[4] = algorithm + * data[5] = AAD LSB address + * data[6] = Reserved[31:16] | AAD_size[15:0] + * data[7] = Input LSB address + * data[8] = Output LSB address + * data[9] = Input size (u32) + * data[10] = Output size (u32) + * data[11] = CRC + * IV size is the 16-bit low half of data[3]; AAD size is the 16-bit low half + * of data[6]; input and output sizes are full u32 words. + */ +static const struct se_cmd_addr_field ele_auth_enc_addr_fields[] = { + /* iv_address (size[15:0] @ data[3]) */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_mask = 0xFFFFu }, + /* aad_address (size[15:0] @ data[6]) */ + { .lsb_idx = 5, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 6, .size_mask = 0xFFFFu }, + /* input_address (size[31:0] @ data[9]) */ + { .lsb_idx = 7, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFFFFFu }, + /* output_address (size[31:0] @ data[10]) */ + { .lsb_idx = 8, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 10, .size_mask = 0xFFFFFFFFu }, +}; + +/* ELE_AUTH_ENC_NEW_REQ: AEAD encrypt/decrypt with internally-generated IV output. */ +#define ELE_AUTH_ENC_IV_OUT_SIZE 12 /* firmware always writes exactly 12 bytes */ +static const struct se_cmd_addr_field ele_auth_enc_new_addr_fields[] = { + /* iv_address_in length is packed in the high half of the iv-size word */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_shift = 16, .size_mask = 0xFFFFu }, /* iv_address_in */ + /* iv_address_out has a fixed firmware-defined length, not carried in msg */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = SE_CMD_ADDR_FIXED_SIZE, + .buf_size = ELE_AUTH_ENC_IV_OUT_SIZE }, /* iv_address_out */ + /* key: plaintext-key buffer only when the PLAINTEXT_KEY flag is set */ + { .lsb_idx = 5, .flag_idx = 2, .flag_mask = 0x00000008u, .is_addr_when_set = true, + .size_idx = 7, .size_shift = 16, .size_mask = 0xFFFFu }, /* key_size */ + { .lsb_idx = 6, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 7, .size_mask = 0xFFFFu }, /* tag_address */ + { .lsb_idx = 8, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFFFFFu }, /* aad_address */ + { .lsb_idx = 10, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 11, .size_mask = 0xFFFFFFFFu }, /* input_address */ + { .lsb_idx = 12, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 13, .size_mask = 0xFFFFFFFFu }, /* output_address */ + { .lsb_idx = 14, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 15, .size_mask = 0xFFFFu }, /* context_address */ +}; + +/* ELE_SIGNATURE_GENERATE_REQ: digital signature generation. */ +static const struct se_cmd_addr_field ele_sign_gen_addr_fields[] = { + /* key: plaintext-key buffer only when GENERATE_SIGN PLAINTEXT_KEY is set */ + { .lsb_idx = 1, .flag_idx = 5, .flag_mask = 0x00080000u, .is_addr_when_set = true, + .size_idx = 8, .size_mask = 0xFFFFu }, /* priv_key_size */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* message_addr */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFu }, /* signature_addr */ + { .lsb_idx = 9, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 11, .size_mask = 0xFFFFu }, /* sm2_pub_key_addr */ + { .lsb_idx = 10, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 11, .size_shift = 16, + .size_mask = 0xFFFFu }, /* sm2_id / ml_dsa_ctx addr */ +}; + +static const struct se_cmd_addr_field ele_pub_key_attest_addr_fields[] = { + /* + * PSA_COMPLIANT public-key-attestation payload. sig_gen_hdl is data[0], + * key_identifier data[1], key_attestation_id data[2], attest_algo + * data[3]. The two buffer addresses that follow are each written + * unconditionally via set_phy_addr_to_words() (single LSB word, high + * half always zero), so both are always DMA addresses. Each address is + * immediately followed by its full u32 byte length. + */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFFFFFu }, /* auth_challenge_addr */ + { .lsb_idx = 6, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 7, .size_mask = 0xFFFFFFFFu }, /* certificate_addr */ +}; + +/* ELE_SIGNATURE_VERIFY_REQ: digital signature verification. */ +static const struct se_cmd_addr_field ele_verify_sign_addr_fields[] = { + /* key: plaintext-key buffer unless the VERIFY_SIGN OPAQUE_KEY flag is set */ + { .lsb_idx = 1, .flag_idx = 7, .flag_mask = 0x00000008u, .is_addr_when_set = false, + .size_idx = 5, .size_shift = 16, .size_mask = 0xFFFFu }, /* key_size */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* msg_addr */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFu }, /* sig_addr */ + { .lsb_idx = 10, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 11, .size_mask = 0xFFFFu }, /* sm2_id / ml_dsa_ctx addr */ +}; + +static const struct se_cmd_addr_field ele_data_storage_addr_fields[] = { + /* + * PSA_COMPLIANT data-storage payload. data_storage_handle is data[0], + * flags/reserved is data[1], data_id is data[2]. data_address (data[3]) + * is the plaintext data buffer, written unconditionally via + * set_phy_addr_to_words() (single LSB word, high half always zero), so + * it is always a DMA address. Its length is the full u32 data_size in + * data[4]. + */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* data_address */ +}; + +/* ELE_ASYMMETRIC_ENC_REQ: asymmetric encryption/decryption. */ +static const struct se_cmd_addr_field ele_asym_enc_addr_fields[] = { + /* key_id_addr: plaintext-key buffer only when the PLAINTEXT_KEY flag is set */ + { .lsb_idx = 1, .flag_idx = 8, .flag_mask = 0x00000008u, .is_addr_when_set = true, + .size_idx = 10, .size_mask = 0xFFFFFFFFu }, /* input_plainkey_size */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFFFFFu }, /* plaintext_addr */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 6, .size_mask = 0xFFFFFFFFu }, /* ciphertext_addr */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 7, .size_mask = 0xFFFFFFFFu }, /* label_addr */ +}; + +/* ELE_KEY_GENERIC_CRYPTO_REQ: generic crypto operation with a raw key. */ +static const struct se_cmd_addr_field ele_key_generic_crypto_addr_fields[] = { + /* key_address length is the u8 key_size in the third byte of the iv-size word */ + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_shift = 16, .size_mask = 0xFFu }, /* key_address */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_mask = 0xFFFFu }, /* iv_address */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFu }, /* aad_address */ + { .lsb_idx = 6, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 8, .size_mask = 0xFFFFFFFFu }, /* input_address */ + { .lsb_idx = 7, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFFFFFu }, /* output_address */ +}; + +/* ELE_GC_CIPHER_REQ: GC symmetric cipher operation. */ +static const struct se_cmd_addr_field ele_gc_cipher_addr_fields[] = { + { .lsb_idx = 0, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFFFFFu }, /* in_addr */ + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFFFFFu }, /* out_addr (shares data_size) */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* key_addr */ + { .lsb_idx = 5, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 6, .size_mask = 0xFFFFFFFFu }, /* iv_addr */ +}; + +/* ELE_GC_AEAD_REQ: GC AEAD operation. */ +static const struct se_cmd_addr_field ele_gc_aead_addr_fields[] = { + { .lsb_idx = 0, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFFFFFu }, /* in_addr */ + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFFFFFu }, /* out_addr (shares data_size) */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* key_addr */ + { .lsb_idx = 5, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 6, .size_mask = 0xFFFFFFFFu }, /* nonce_addr */ + { .lsb_idx = 7, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 8, .size_mask = 0xFFFFFFFFu }, /* aad_addr */ + { .lsb_idx = 9, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 10, .size_mask = 0xFFFFFFFFu }, /* tag_addr */ +}; + +/* ELE_GC_ACRYPTO_REQ: GC asymmetric crypto operation. */ +static const struct se_cmd_addr_field ele_gc_acrypto_addr_fields[] = { + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFFFFFu }, /* data_buff1_addr */ + { .lsb_idx = 4, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 6, .size_mask = 0xFFFFFFFFu }, /* data_buff2_addr */ + { .lsb_idx = 7, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFu }, /* key_buff1_addr */ + { .lsb_idx = 8, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_shift = 16, .size_mask = 0xFFFFu }, /* key_buff2_addr */ + { .lsb_idx = 12, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 13, .size_mask = 0xFFFFu }, /* rsa_label_addr */ +}; + +/* ELE_GC_AKEY_GEN_REQ: GC asymmetric key generation. */ +static const struct se_cmd_addr_field ele_gc_akey_gen_addr_fields[] = { + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFu }, /* modulus_addr */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_shift = 16, .size_mask = 0xFFFFu }, /* priv_buff_addr */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFu }, /* pub_buff_addr */ +}; + +/* ELE_HASH_ONE_GO_REQ: hash one-go operation. */ +static const struct se_cmd_addr_field ele_hash_one_go_addr_fields[] = { + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 6, .size_shift = 16, .size_mask = 0xFFFFu }, /* ctx_addr */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 4, .size_mask = 0xFFFFFFFFu }, /* input_addr */ + { .lsb_idx = 3, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 5, .size_mask = 0xFFFFFFFFu }, /* output_addr */ +}; + +static const struct se_cmd_addr_field ele_enc_data_storage_addr_fields[] = { + /* + * PSA_COMPLIANT encrypted-data-storage payload. data_storage_handle is + * data[0] and data_id is data[1]. data_address (data[2]) is written + * unconditionally via set_phy_addr_to_words() (single LSB word, high + * half always zero), so it is always a DMA address; its length is the + * full u32 data_size in data[3]. iv_address (data[8]) is only written + * when an IV is supplied and is left zero otherwise, so it is an + * optional always-address handled by the zero-address skip; its length + * is the u16 iv_size in the low half of data[9]. + */ + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 3, .size_mask = 0xFFFFFFFFu }, /* data_address */ + { .lsb_idx = 8, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 9, .size_mask = 0xFFFFu }, /* iv_address */ +}; + +static const struct se_cmd_addr_field ele_rng_get_random_addr_fields[] = { + /* + * PSA_COMPLIANT get-random payload. reserved/flags is data[0]; rnd_addr + * (data[1]) is the output buffer, written unconditionally via + * set_phy_addr_to_words() (single LSB word, high half always zero), so + * it is always a DMA address. Its length is the full u32 rnd_size in + * data[2]. + */ + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFFFFFu }, /* rnd_addr */ +}; + +static const struct se_cmd_addr_field ele_storage_master_import_addr_fields[] = { + /* + * Storage master-import command (ELE_STORAGE_MASTER_IMPORT_REQ). + * Payload layout (data[] = message word minus header word 0): + * data[0] = storage_handle + * data[1] = key_store_address (LSB; high half always zero) + * data[2] = key_store_size + * The address is set unconditionally via set_phy_addr_to_words() and + * its length is the full u32 key_store_size. + */ + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 2, .size_mask = 0xFFFFFFFFu }, /* key_store_address */ +}; + +const struct se_cmd_addr_field *ele_fw_cmd_addr_fields(u8 cmd, size_t *count) +{ + switch (cmd) { + case ELE_PUB_KEY_EXPORT_REQ: + *count = ARRAY_SIZE(ele_pub_key_export_addr_fields); + return ele_pub_key_export_addr_fields; + case ELE_KEYSTORE_REPROV_ENABLE_REQ: + *count = ARRAY_SIZE(ele_keystore_reprov_en_addr_fields); + return ele_keystore_reprov_en_addr_fields; + case ELE_KEYGEN_REQ: + *count = ARRAY_SIZE(ele_keygen_addr_fields); + return ele_keygen_addr_fields; + case ELE_KEY_EXCHANGE_REQ: + *count = ARRAY_SIZE(ele_key_exchange_addr_fields); + return ele_key_exchange_addr_fields; + case ELE_KEY_IMPORT_REQ: + *count = ARRAY_SIZE(ele_key_import_addr_fields); + return ele_key_import_addr_fields; + case ELE_MAC_REQ: + *count = ARRAY_SIZE(ele_mac_addr_fields); + return ele_mac_addr_fields; + case ELE_CIPHER_REQ: + *count = ARRAY_SIZE(ele_cipher_addr_fields); + return ele_cipher_addr_fields; + case ELE_AUTH_ENC_REQ: + *count = ARRAY_SIZE(ele_auth_enc_addr_fields); + return ele_auth_enc_addr_fields; + case ELE_AUTH_ENC_NEW_REQ: + *count = ARRAY_SIZE(ele_auth_enc_new_addr_fields); + return ele_auth_enc_new_addr_fields; + case ELE_SIGNATURE_GENERATE_REQ: + *count = ARRAY_SIZE(ele_sign_gen_addr_fields); + return ele_sign_gen_addr_fields; + case ELE_PUB_KEY_ATTEST_REQ: + *count = ARRAY_SIZE(ele_pub_key_attest_addr_fields); + return ele_pub_key_attest_addr_fields; + case ELE_SIGNATURE_VERIFY_REQ: + *count = ARRAY_SIZE(ele_verify_sign_addr_fields); + return ele_verify_sign_addr_fields; + case ELE_DATA_STORAGE_REQ: + *count = ARRAY_SIZE(ele_data_storage_addr_fields); + return ele_data_storage_addr_fields; + case ELE_ENC_DATA_STORAGE_REQ: + *count = ARRAY_SIZE(ele_enc_data_storage_addr_fields); + return ele_enc_data_storage_addr_fields; + case ELE_ASYMMETRIC_ENC_REQ: + *count = ARRAY_SIZE(ele_asym_enc_addr_fields); + return ele_asym_enc_addr_fields; + case ELE_KEY_GENERIC_CRYPTO_REQ: + *count = ARRAY_SIZE(ele_key_generic_crypto_addr_fields); + return ele_key_generic_crypto_addr_fields; + case ELE_GC_CIPHER_REQ: + *count = ARRAY_SIZE(ele_gc_cipher_addr_fields); + return ele_gc_cipher_addr_fields; + case ELE_GC_AEAD_REQ: + *count = ARRAY_SIZE(ele_gc_aead_addr_fields); + return ele_gc_aead_addr_fields; + case ELE_GC_ACRYPTO_REQ: + *count = ARRAY_SIZE(ele_gc_acrypto_addr_fields); + return ele_gc_acrypto_addr_fields; + case ELE_GC_AKEY_GEN_REQ: + *count = ARRAY_SIZE(ele_gc_akey_gen_addr_fields); + return ele_gc_akey_gen_addr_fields; + case ELE_HASH_ONE_GO_REQ: + *count = ARRAY_SIZE(ele_hash_one_go_addr_fields); + return ele_hash_one_go_addr_fields; + case ELE_RNG_GET_RANDOM_REQ: + *count = ARRAY_SIZE(ele_rng_get_random_addr_fields); + return ele_rng_get_random_addr_fields; + case ELE_STORAGE_MASTER_IMPORT_REQ: + *count = ARRAY_SIZE(ele_storage_master_import_addr_fields); + return ele_storage_master_import_addr_fields; + default: + *count = 0; + return NULL; + } +} + +/* + * FW API for Command Receiver. + * + * Storage master-export response (ELE_STORAGE_MASTER_EXPORT_REQ). + * The cmd_receiver sends this response to firmware to supply the + * output buffer address. Payload layout: + * data[0] = storage_handle + * data[1] = rsp_code + * data[2] = key_store_export_address (LSB; high half always zero) + * No length word is present in the response itself; the export size is + * taken from the FW command received earlier (key_store_size) and stored + * in se_if_priv.cmd_rcvr_var_size by cmd_receiver_specific_ops(). + * se_val_cmd_addrs() reads it when size_idx == SE_CMD_RCVR_ADDR_VAR_SIZE + * to range-check the full response buffer before it is forwarded to FW. + */ +static const struct se_cmd_addr_field ele_storage_master_export_addr_fields[] = { + { .lsb_idx = 2, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = SE_CMD_RCVR_ADDR_VAR_SIZE }, /* key_store_export_address */ +}; + +/* + * Storage chunk-get response (ELE_STORAGE_CHUNK_GET_REQ). + * The cmd_receiver fills in the chunk buffer address and its size so + * firmware can DMA the chunk data into the kernel's coherent buffer. + * Payload layout: + * data[0] = chunk_size + * data[1] = chunk_addr (LSB; high half always zero) + * data[2] = rsp_code + * The size word precedes the address in the message. + */ +static const struct se_cmd_addr_field ele_storage_chunk_get_addr_fields[] = { + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = 0, .size_mask = 0xFFFFFFFFu }, /* chunk_addr */ +}; + +/* + * Storage chunk-export response (ELE_STORAGE_CHUNK_EXPORT_REQ). + * The cmd_receiver supplies the output buffer address. Payload layout: + * data[0] = rsp_code + * data[1] = chunk_export_address (LSB; high half always zero) + * No length word is present in the response itself; the export size is + * taken from the FW command received earlier (chunk_size) and stored + * in se_if_priv.cmd_rcvr_var_size by cmd_receiver_specific_ops(). + * se_val_cmd_addrs() reads it when size_idx == SE_CMD_RCVR_ADDR_VAR_SIZE + * to range-check the full response buffer before it is forwarded to FW. + */ +static const struct se_cmd_addr_field ele_storage_chunk_export_addr_fields[] = { + { .lsb_idx = 1, .flag_idx = SE_CMD_ADDR_ALWAYS, + .size_idx = SE_CMD_RCVR_ADDR_VAR_SIZE }, /* chunk_export_address */ +}; + +/* + * Return the address-field descriptor table for a cmd_receiver response + * message (rsp_tag), or NULL when the response embeds no DMA addresses. + * count is set to the number of entries. Only the three storage responses + * that supply a kernel buffer address to firmware are covered here; + * ELE_STORAGE_EXPORT_FINISH_REQ, ELE_STORAGE_CHUNK_GET_DONE_REQ, and + * ELE_STORAGE_CHUNK_DELETE_REQ carry no DMA addresses and return NULL. + */ +const struct se_cmd_addr_field *ele_fw_rsp_addr_fields(u8 cmd, size_t *count) +{ + switch (cmd) { + case ELE_STORAGE_MASTER_EXPORT_REQ: + *count = ARRAY_SIZE(ele_storage_master_export_addr_fields); + return ele_storage_master_export_addr_fields; + case ELE_STORAGE_CHUNK_GET_REQ: + *count = ARRAY_SIZE(ele_storage_chunk_get_addr_fields); + return ele_storage_chunk_get_addr_fields; + case ELE_STORAGE_CHUNK_EXPORT_REQ: + *count = ARRAY_SIZE(ele_storage_chunk_export_addr_fields); + return ele_storage_chunk_export_addr_fields; + default: + *count = 0; + return NULL; + } +} diff --git a/drivers/firmware/imx/se_ctrl.c b/drivers/firmware/imx/se_ctrl.c index b5d3421c0987..3b1209325633 100644 --- a/drivers/firmware/imx/se_ctrl.c +++ b/drivers/firmware/imx/se_ctrl.c @@ -4,6 +4,7 @@ */ #include +#include #include #include #include @@ -15,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -24,18 +26,17 @@ #include #include #include +#include #include "ele_base_msg.h" #include "ele_common.h" +#include "ele_fw_api.h" #include "se_ctrl.h" -#define MAX_SOC_INFO_DATA_SZ 256 -#define SE_TYPE_STR_DBG "dbg" -#define SE_TYPE_STR_HSM "hsm" - -#define SE_TYPE_ID_DBG 0x1 +/* Maximum response buffer size in bytes for debug-dump replies. */ +#define MAX_ALLOWED_TX_MSG_SZ SZ_4K -#define SE_TYPE_ID_HSM 0x2 +#define MAX_SOC_INFO_DATA_SZ 256 struct se_soc_dev_regn { bool soc_dev_registered; @@ -138,6 +139,13 @@ char *get_se_if_name(u8 se_if_id) return "unknown"; } +static u32 get_se_soc_id(struct se_if_priv *priv) +{ + const struct se_if_node *if_node = device_get_match_data(priv->dev); + + return if_node->se_info->soc_id; +} + static struct se_fw_load_info *get_load_fw_instance(struct se_if_priv *priv) { return &priv->load_fw; @@ -294,15 +302,325 @@ static int get_se_soc_info(struct se_if_priv *priv, const struct se_soc_info *se return 0; } +static int load_firmware(struct se_if_priv *priv, const u8 *se_img_file_to_load) +{ + const struct firmware *fw = NULL; + dma_addr_t se_fw_dma_addr; + u32 se_fw_buf_len; + void *se_fw_buf; + int ret; + + if (!se_img_file_to_load) { + dev_err(priv->dev, "FW image is not provided.\n"); + return -EINVAL; + } + ret = request_firmware(&fw, se_img_file_to_load, priv->dev); + if (ret) + return ret; + + if (fw->size > U32_MAX) { + ret = -EFBIG; + release_firmware(fw); + return ret; + } + dev_info(priv->dev, "loading firmware %s.\n", se_img_file_to_load); + + /* + * Serialize access to priv_dev_ctx shared memory to prevent pos + * corruption if two driver-internal callers run concurrently (e.g. + * ele_get_info() racing with load_firmware()). + */ + scoped_guard(mutex, &priv->priv_dev_ctx->fops_lock) { + se_fw_buf_len = fw->size; + ret = get_shared_mem_slot(priv->priv_dev_ctx, + &se_fw_buf_len, &se_fw_dma_addr, + &se_fw_buf); + if (ret) { + dev_err(priv->dev, "Failed to allocate firmware shared buffer: %d\n", + ret); + release_firmware(fw); + return ret; + } + + memcpy(se_fw_buf, fw->data, fw->size); + ret = ele_fw_authenticate(priv, se_fw_dma_addr, se_fw_dma_addr); + if (ret < 0) { + dev_err(priv->dev, + "Error %pe: Authenticate & load SE firmware %s.", + ERR_PTR(ret), se_img_file_to_load); + ret = -EPERM; + } + if (!se_is_fw_busy_ctx(priv->priv_dev_ctx)) + se_dev_ctx_shared_mem_cleanup(priv->priv_dev_ctx); + } + + release_firmware(fw); + + return ret; +} + +static int se_load_firmware(struct se_if_priv *priv) +{ + struct se_fw_load_info *load_fw = get_load_fw_instance(priv); + int ret = 0; + + guard(mutex)(&load_fw->load_fw_lock); + if (!load_fw->is_fw_tobe_loaded) + return 0; + + if (load_fw->imem.state == ELE_IMEM_STATE_BAD) { + ret = load_firmware(priv, load_fw->se_fw_img_nm->prim_fw_nm_in_rfs); + if (ret) { + dev_err(priv->dev, "Failed to load boot firmware.\n"); + return -EPERM; + } + } + + ret = load_firmware(priv, load_fw->se_fw_img_nm->seco_fw_nm_in_rfs); + if (ret) { + dev_err(priv->dev, "Failed to load runtime firmware.\n"); + return -EPERM; + } + + load_fw->is_fw_tobe_loaded = false; + + return ret; +} + +static int init_se_shared_mem(struct se_if_device_ctx *dev_ctx) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct se_if_priv *priv = dev_ctx->priv; + + INIT_LIST_HEAD(&se_shared_mem_mgmt->pending_out); + INIT_LIST_HEAD(&se_shared_mem_mgmt->pending_in); + + if (priv->mem_pool) + INIT_LIST_HEAD(&se_shared_mem_mgmt->mem_pool_buf_list); + + se_shared_mem_mgmt->non_secure_mem.ptr = + dma_alloc_coherent(priv->dev, MAX_DATA_SIZE_PER_USER, + &se_shared_mem_mgmt->non_secure_mem.dma_addr, + GFP_KERNEL); + if (!se_shared_mem_mgmt->non_secure_mem.ptr) + return -ENOMEM; + + se_shared_mem_mgmt->non_secure_mem.size = MAX_DATA_SIZE_PER_USER; + se_shared_mem_mgmt->non_secure_mem.pos = 0; + + return 0; +} + +static void cleanup_se_shared_mem(struct se_if_device_ctx *dev_ctx, bool reclaim) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct se_if_priv *priv = dev_ctx->priv; + bool free_dma_buf; + + /* + * mem_pool_buf_list is only initialised for interfaces that own a + * gen_pool (priv->mem_pool != NULL). On interfaces without a pool + * (e.g. imx93, which has no pool_name) the list head is left + * zero-filled, so se_cleanup_mem_pool_buf() must not walk it here or + * list_for_each_entry_safe() would dereference a NULL head and panic + * the kernel on close/teardown. Skip the pool cleanup entirely when + * there is no pool; there is nothing to reclaim in that case. + */ + if (priv->mem_pool) + se_cleanup_mem_pool_buf(dev_ctx, reclaim); + + /* Guard against being called before shared memory was ever allocated + * (e.g. probe failure before dma_alloc_coherent succeeded). + */ + if (!se_shared_mem_mgmt->non_secure_mem.ptr) + return; + + /* + * Decide whether the DMA buffer can be released before touching the + * pending lists. se_dev_ctx_shared_mem_cleanup() resets + * non_secure_mem.pos, so the "nothing staged" test must be sampled + * here first. When reclaim is false the buffer is released only if no + * data is still staged for the firmware; otherwise the enclave may + * still be DMA-ing into it and the buffer is deliberately leaked to + * avoid a DMA-after-free. + */ + free_dma_buf = reclaim || !se_shared_mem_mgmt->non_secure_mem.pos; + + /* + * Free any se_buf_desc items that were never consumed (e.g. when the + * fd is closed while pending I/O buffers are still listed). This must + * happen before the DMA backing memory is released to avoid a leak. + */ + se_dev_ctx_shared_mem_cleanup(dev_ctx); + + if (free_dma_buf) { + dma_free_coherent(priv->dev, MAX_DATA_SIZE_PER_USER, + se_shared_mem_mgmt->non_secure_mem.ptr, + se_shared_mem_mgmt->non_secure_mem.dma_addr); + } + + /* + * Drop the host-side tracking unconditionally. On the reclaim path the + * buffer has been freed. On the deliberate-leak path the buffer is + * abandoned on purpose, so clearing the pointer here guarantees a later + * cleanup pass (e.g. se_if_priv_release()) cannot double-free it. + */ + se_shared_mem_mgmt->non_secure_mem.ptr = NULL; + se_shared_mem_mgmt->non_secure_mem.dma_addr = 0; + se_shared_mem_mgmt->non_secure_mem.size = 0; + se_shared_mem_mgmt->non_secure_mem.pos = 0; +} + +static int se_dev_ctx_cpy_out_data(struct se_if_device_ctx *dev_ctx) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct se_if_priv *priv = dev_ctx->priv; + struct se_buf_desc *b_desc, *temp; + bool do_cpy = true; + + list_for_each_entry_safe(b_desc, temp, &se_shared_mem_mgmt->pending_out, link) { + if (b_desc->usr_buf_ptr && b_desc->shared_buf_ptr && do_cpy) { + dev_dbg(priv->dev, "Copying output data to user.\n"); + if (do_cpy && copy_to_user(b_desc->usr_buf_ptr, + b_desc->shared_buf_ptr, + b_desc->size)) { + dev_err(priv->dev, "Failure copying output data to user.\n"); + do_cpy = false; + } + } + + if (b_desc->shared_buf_ptr) + memset(b_desc->shared_buf_ptr, 0, b_desc->size); + + list_del(&b_desc->link); + kfree(b_desc); + } + + return do_cpy ? 0 : -EFAULT; +} + +/* + * Clean the used Shared Memory space, + * whether its Input Data copied from user buffers, or + * Data received from FW. + */ +void se_dev_ctx_shared_mem_cleanup(struct se_if_device_ctx *dev_ctx) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct list_head *pending_lists[] = {&se_shared_mem_mgmt->pending_in, + &se_shared_mem_mgmt->pending_out}; + struct se_buf_desc *b_desc, *temp; + bool is_fw_busy_dev_ctx; + int i; + + /* + * If this context is the one that caused a firmware timeout the shared + * DMA buffers may still be actively read/written by the firmware. + */ + is_fw_busy_dev_ctx = se_is_fw_busy_ctx(dev_ctx); + + for (i = 0; i < ARRAY_SIZE(pending_lists); i++) { + list_for_each_entry_safe(b_desc, temp, pending_lists[i], link) { + if (!is_fw_busy_dev_ctx && b_desc->shared_buf_ptr) + memset(b_desc->shared_buf_ptr, 0, b_desc->size); + + list_del(&b_desc->link); + kfree(b_desc); + } + } + + /* + * Keep non_secure_mem.pos non-zero while this context still owns an + * outstanding firmware transaction. A non-zero pos is the marker that + * data is still staged for the enclave, which cleanup_se_shared_mem() + * uses to decide the buffer must be leaked rather than freed. Resetting + * it here would let a later teardown pass free a buffer the enclave may + * still be DMA-ing into. + */ + if (!is_fw_busy_dev_ctx) + se_shared_mem_mgmt->non_secure_mem.pos = 0; +} + +static struct se_buf_desc *add_b_desc_to_pending_list(void *shared_ptr_with_pos, + struct se_ioctl_setup_iobuf *io, + struct se_if_device_ctx *dev_ctx) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct se_buf_desc *b_desc = NULL; + + b_desc = kzalloc_obj(*b_desc); + if (!b_desc) + return ERR_PTR(-ENOMEM); + + b_desc->shared_buf_ptr = shared_ptr_with_pos; + b_desc->usr_buf_ptr = u64_to_user_ptr(io->user_buf); + b_desc->size = io->length; + + if (io->flags & SE_IO_BUF_FLAGS_IS_INPUT) { + /* + * buffer is input: + * add an entry in the "pending input buffers" list so + * that copied data can be cleaned from shared memory + * later. + */ + list_add_tail(&b_desc->link, &se_shared_mem_mgmt->pending_in); + } else { + /* + * buffer is output: + * add an entry in the "pending out buffers" list so data + * can be copied to user space when receiving Secure-Enclave + * response. + */ + list_add_tail(&b_desc->link, &se_shared_mem_mgmt->pending_out); + } + + return b_desc; +} + +static void se_if_open_gate_release(struct kref *kref) +{ + struct se_if_open_gate *gate = + container_of(kref, struct se_if_open_gate, refcount); + + kfree(gate); +} + +static bool se_if_open_gate_get(struct se_if_open_gate *gate) +{ + if (!gate) + return false; + + return kref_get_unless_zero(&gate->refcount); +} + +static void se_if_open_gate_put(struct se_if_open_gate *gate) +{ + if (gate) + kref_put(&gate->refcount, se_if_open_gate_release); +} + +/* + * Distinct lockdep class for the internal priv_dev_ctx fops_lock. Taking it + * while an open context's fops_lock is held (for example a firmware load + * triggered from an ioctl) is valid hierarchical locking, but shares the same + * class as the per-open fops_lock and would otherwise be misreported as + * recursive locking by lockdep. + */ +static struct lock_class_key se_priv_ctx_fops_key; + static int init_misc_device_context(struct se_if_priv *priv, int ch_id, - struct se_if_device_ctx **new_dev_ctx) + struct se_if_device_ctx **new_dev_ctx, + const struct file_operations *se_if_fops) { + struct se_if_open_gate *gate = NULL; struct se_if_device_ctx *dev_ctx; + int ret = -ENOMEM; dev_ctx = kzalloc_obj(*dev_ctx); if (!dev_ctx) return -ENOMEM; + dev_ctx->priv = priv; dev_ctx->devname = kasprintf(GFP_KERNEL, "%s0_ch%d", get_se_if_name(priv->if_defs->se_if_type), ch_id); @@ -311,10 +629,53 @@ static int init_misc_device_context(struct se_if_priv *priv, int ch_id, return -ENOMEM; } - dev_ctx->priv = priv; + mutex_init(&dev_ctx->fops_lock); + lockdep_set_class(&dev_ctx->fops_lock, &se_priv_ctx_fops_key); + + kref_init(&dev_ctx->refcount); + dev_ctx->cleanup_done = false; *new_dev_ctx = dev_ctx; + set_se_rcv_msg_timeout(dev_ctx, SE_RCV_MSG_DEFAULT_TIMEOUT_MS); + + ret = init_se_shared_mem(dev_ctx); + if (ret < 0) + goto exit; + + gate = kzalloc_obj(*gate); + if (!gate) { + ret = -ENOMEM; + goto exit; + } + + mutex_init(&gate->lock); + kref_init(&gate->refcount); /* device-owned reference */ + gate->priv = priv; + gate->dying = false; + priv->open_gate = gate; + + /* + * The miscdevice storage is now owned by the open gate object. + * priv->priv_dev_ctx still keeps a pointer to that miscdevice. + */ + dev_ctx->miscdev = &gate->miscdev; + + dev_ctx->miscdev->name = dev_ctx->devname; + dev_ctx->miscdev->minor = MISC_DYNAMIC_MINOR; + dev_ctx->miscdev->fops = se_if_fops; + dev_ctx->miscdev->parent = priv->dev; return 0; +exit: + *new_dev_ctx = NULL; + + if (gate) { + priv->open_gate = NULL; + se_if_open_gate_put(gate); + } + cleanup_se_shared_mem(dev_ctx, true); + kfree(dev_ctx->devname); + kfree(dev_ctx); + return ret; } static int se_if_request_channel(struct device *dev, struct mbox_chan **chan, @@ -332,43 +693,236 @@ static int se_if_request_channel(struct device *dev, struct mbox_chan **chan, return 0; } +/* + * Forward declarations. se_if_probe_cleanup() and se_if_probe() are kept + * together as the teardown/probe pair, but several helpers, the file + * operations table and the firmware-busy work handler they reference are + * defined further down in this file. + */ +static void dlink_dev_ctx(struct se_if_device_ctx *dev_ctx); +static void cleanup_dev_ctx(struct se_if_device_ctx *dev_ctx, bool is_fclose); +static void se_clear_fw_busy(struct se_if_priv *priv); +static void se_if_dev_ctx_release(struct kref *kref); +static void se_if_priv_release(struct kref *kref); +static int se_if_misc_register(struct se_if_priv *priv); +static void se_fw_busy_work(struct work_struct *work); +static const struct file_operations se_if_fops; + static void se_if_probe_cleanup(void *plat_dev) { struct platform_device *pdev = plat_dev; + struct se_if_device_ctx *dev_ctx; struct device *dev = &pdev->dev; + struct fw_busy_info *fbusy_info; struct se_if_priv *priv; priv = dev_get_drvdata(dev); if (!priv) return; - if (priv->rx_chan) - mbox_free_channel(priv->rx_chan); - if (priv->tx_chan) - mbox_free_channel(priv->tx_chan); + fbusy_info = &priv->fw_busy_info; /* - * Being device managed buffer, no need to free the buffer allocated - * in se probe to store encrypted IMEM. + * Announce teardown, then wake any in-flight waiter. going_away makes + * ele_msg_send_rcv() bail out instead of arming a new transaction and + * lets ele_msg_rcv() tell a teardown-forced completion apart from a + * real response; it must be set before complete_all(). + * + * Set it under clbk_rx_lock, not se_if_cmd_lock: se_if_cmd_lock is held + * across the whole blocking transaction, so taking it here would stall + * unbind for a full receive-timeout. clbk_rx_lock is the short spinlock + * ele_msg_send_rcv() holds while arming, so this closes the lost-wakeup + * window - the sender either sees going_away and bails before arming, or + * armed first and this store (and complete_all()) is ordered after its + * reinit_completion() - and supplies the ordering the relaxed atomics do + * not. + */ + scoped_guard(spinlock_irqsave, &priv->waiting_rsp_clbk_hdl.clbk_rx_lock) + atomic_set(&priv->going_away, 1); + /* + * Wake the waiter before iterating the device-context list. It sleeps on + * this completion holding dev_ctx->fops_lock, which cleanup_dev_ctx() + * below also takes, so completing first avoids an unbind hang. Runs + * outside clbk_rx_lock; the going_away store above already orders it + * against the arming path. */ + complete_all(&priv->waiting_rsp_clbk_hdl.done); /* - * No need to check, if reserved memory is allocated - * before calling for its release. Or clearing the - * un-set bit. + * Only now reserve the messaging interface for this teardown flow. + * + * se_reserve_msg_if() blocks on msg_excl_flow_lock, and the fw_busy + * recovery worker (se_fw_busy_work() -> se_clear_fw_busy()) may already + * hold that reservation while parked uninterruptibly in ele_msg_rcv() + * waiting on a possibly hung firmware for up to the full receive + * timeout. The only thing that cuts that wait short is the complete_all() + * above, so it MUST run before this reserve: otherwise teardown would + * sleep on the reservation the worker holds, the worker would stay + * blocked on firmware, and unbind would stall for the entire multi- + * thousand-second timeout (an unbind hang / hung-task). + * + * With going_away already set and the in-flight waiter already forced to + * unwind, the worker returns promptly (its send is failed with -ENODEV), + * drops the reservation, and this call acquires it without waiting on + * anything firmware-related. From here on teardown is the exclusive + * owner: ele_msg_send_rcv() lets only this task's priv_dev_ctx close + * traffic through and rejects every other caller. */ - of_reserved_mem_device_release(dev); + se_reserve_msg_if(priv); - dev_set_drvdata(dev, NULL); + /* + * Mark the private device context as cleanup_done first. + * This prevents new device contexts from being created in open(). + */ + if (priv->priv_dev_ctx) { + /* + * Mark cleanup_done under fops_lock so that se_if_fops_open(), + * which checks cleanup_done while holding fops_lock, cannot + * race past this and add a new device context after teardown. + */ + scoped_guard(mutex, &priv->priv_dev_ctx->fops_lock) + priv->priv_dev_ctx->cleanup_done = true; + + if (priv->open_gate) { + scoped_guard(mutex, &priv->open_gate->lock) { + priv->open_gate->dying = true; + priv->open_gate->priv = NULL; + } + } + + /* + * misc_register() is deferred to the end of probe, so the + * device may have a miscdev set up but never registered if + * probe failed before se_if_misc_register(). Only deregister + * when registration actually succeeded. + */ + if (priv->open_gate && priv->open_gate->registered && + priv->priv_dev_ctx->miscdev) + misc_deregister(priv->priv_dev_ctx->miscdev); + } + + while (true) { + bool list_was_empty = false; + + dev_ctx = NULL; + + scoped_guard(mutex, &priv->modify_lock) { + if (list_empty(&priv->dev_ctx_list)) { + list_was_empty = true; + } else { + dev_ctx = list_first_entry(&priv->dev_ctx_list, + struct se_if_device_ctx, link); + + /* pin this context so close() cannot free it under us */ + kref_get(&dev_ctx->refcount); + dlink_dev_ctx(dev_ctx); + } + } + + if (list_was_empty) + break; + + /* + * Local cleanup outside the global lock avoids ABBA deadlock + * with paths that already take dev_ctx->fops_lock first. + */ + cleanup_dev_ctx(dev_ctx, false); + kref_put(&dev_ctx->refcount, se_if_dev_ctx_release); + } + + se_release_msg_if(priv); + /* + * Release any dev_ctx retained by the firmware-busy circuit breaker. + * A synchronous command that timed out parks its dev_ctx in + * fbusy_info->fw_busy_dev_ctx so that a late firmware response can still be + * routed back. If no such response arrived before teardown, that + * retained reference must be dropped here to avoid a leak. + * se_clear_fw_busy() is safe to call unconditionally: it checks + * fbusy_info->fw_busy_dev_ctx under fw_busy_lock and is a no-op when + * nothing is parked. + */ + se_clear_fw_busy(priv); + + /* + * Free the mailbox channels under se_if_cmd_lock. ele_msg_send_rcv() + * holds se_if_cmd_lock for the full duration of a synchronous + * transaction, including the mbox_send_message() call on priv->tx_chan. + * going_away was set above and complete_all() has already woken any + * in-flight waiter, so any transaction in progress will unwind to + * -ENODEV and release the lock quickly. Acquiring se_if_cmd_lock here + * guarantees no caller is still touching the channels when they are + * freed, and nulling the pointers under the lock prevents any sender + * that races past the going_away check from accessing a freed channel. + */ + scoped_guard(mutex, &priv->se_if_cmd_lock) { + if (priv->rx_chan) { + mbox_free_channel(priv->rx_chan); + priv->rx_chan = NULL; + } + if (priv->tx_chan) { + mbox_free_channel(priv->tx_chan); + priv->tx_chan = NULL; + } + } + + /* + * Cancel any pending fw_busy_work before dropping the initial priv + * reference. going_away was set above, so no new work can be scheduled + * after this point. Canceling here while probe_cleanup still holds its + * own priv reference prevents two races: + * + * 1. UAF: if fw_busy_work has dev_ctx == priv_dev_ctx, letting it run + * past this point while se_if_priv_release() frees priv_dev_ctx + * causes a use-after-free of dev_ctx->fops_lock in se_clear_fw_busy(). + * + * 2. Deadlock: if fw_busy_work drops the last priv reference, + * se_if_dev_ctx_release() -> se_if_priv_release() would call + * cancel_work_sync() from inside the worker, causing the worker to + * wait for its own completion. + * + * Both are avoided by canceling here: probe_cleanup still holds a priv + * reference so the worker cannot invoke se_if_priv_release(), and the + * cancel runs from a non-worker context. + */ + cancel_work_sync(&fbusy_info->fw_busy_work); + /* + * Reclaim priv_dev_ctx shared memory before of_reserved_mem_device_release(): + * cleanup_se_shared_mem() calls dma_free_coherent(), which must run while + * the DMA config is still active. fw_busy_work was canceled above, so no + * concurrent caller holds priv_dev_ctx->fops_lock. + * + * reclaim=true is safe even if FW hung at teardown (command timed out, pos + * still non-zero): the ELE region is no-map/shared-dma-pool, so freeing only + * drops the kernel VA/bitmap while the physical pages stay reserved (no + * DMA-after-free). The next probe also sends ELE_GET_INFO into a fresh + * buffer before accepting commands, so a stale FW write to the old buffer is + * never observed by the new driver instance. + */ if (priv->priv_dev_ctx) { - kfree(priv->priv_dev_ctx->devname); - kfree(priv->priv_dev_ctx); + scoped_guard(mutex, &priv->priv_dev_ctx->fops_lock) + cleanup_se_shared_mem(priv->priv_dev_ctx, true); } - mutex_destroy(&priv->load_fw.load_fw_lock); - mutex_destroy(&priv->se_if_cmd_lock); - kfree(priv); + /* + * Release the reserved DMA memory configuration at unbind time, paired + * with of_reserved_mem_device_init() in se_if_probe(). This must not be + * deferred to se_if_priv_release(): that runs when the last file + * descriptor closes, which may be after a new driver instance has already + * called of_reserved_mem_device_init() on the same struct device. Calling + * the release at that point would corrupt the new instance's DMA setup. + */ + of_reserved_mem_device_release(dev); + + /* + * Being device managed buffer, no need to free the buffer allocated + * in se probe to store encrypted IMEM. + */ + + dev_set_drvdata(dev, NULL); + + /* Drop the initial reference - priv will be freed when last fd closes */ + kref_put(&priv->refcount, se_if_priv_release); } static int se_if_probe(struct platform_device *pdev) @@ -391,16 +945,37 @@ static int se_if_probe(struct platform_device *pdev) return -ENOMEM; priv->dev = dev; + /* + * Pin the parent device for the lifetime of priv. A file descriptor may + * stay open after the device is unbound; close() then still passes + * priv->dev to dma_free_coherent()/dev_warn(). Without this reference + * the struct device could be freed while priv->dev still points at it, + * so the reference is dropped in se_if_priv_release() via put_device(). + */ + get_device(priv->dev); + kref_init(&priv->refcount); priv->if_defs = &if_node->if_defs; dev_set_drvdata(dev, priv); spin_lock_init(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock); spin_lock_init(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock); - atomic_set(&priv->fw_busy, 0); + priv->msg_excl_flow.msg_excl_owner = NULL; + spin_lock_init(&priv->msg_excl_flow.msg_excl_lock); + mutex_init(&priv->msg_excl_flow.msg_excl_flow_lock); + struct fw_busy_info *fbusy_info = &priv->fw_busy_info; + + atomic_set(&fbusy_info->fw_busy, 0); + + spin_lock_init(&fbusy_info->fw_busy_lock); + fbusy_info->fw_busy_dev_ctx = NULL; + INIT_WORK(&fbusy_info->fw_busy_work, se_fw_busy_work); + init_completion(&priv->waiting_rsp_clbk_hdl.done); init_completion(&priv->cmd_receiver_clbk_hdl.done); + INIT_LIST_HEAD(&priv->dev_ctx_list); mutex_init(&priv->se_if_cmd_lock); + mutex_init(&priv->modify_lock); load_fw = get_load_fw_instance(priv); mutex_init(&load_fw->load_fw_lock); @@ -455,7 +1030,7 @@ static int se_if_probe(struct platform_device *pdev) load_fw->imem_mgmt = true; } - ret = init_misc_device_context(priv, 0, &priv->priv_dev_ctx); + ret = init_misc_device_context(priv, 0, &priv->priv_dev_ctx, &se_if_fops); if (ret) return dev_err_probe(dev, ret, "Failed to create device contexts.\n"); @@ -466,12 +1041,1309 @@ static int se_if_probe(struct platform_device *pdev) return dev_err_probe(dev, ret, "Failed to fetch SoC Info.\n"); } + /* + * All probe-time initialization is complete; expose the + * interface to userspace last so that an open()/ioctl cannot + * race against a not-yet-initialized device. + */ + ret = se_if_misc_register(priv); + if (ret) + return ret; + dev_info(dev, "i.MX secure-enclave: %s0 interface to firmware, configured.\n", get_se_if_name(priv->if_defs->se_if_type)); return ret; } +/* + * Expose the interface to userspace. Deferred until the end of probe so + * the device node only becomes openable after SoC info has been fetched + * and, on SoCs with IMEM management, the encrypted-IMEM buffer has been + * allocated. This prevents userspace from opening the node and issuing + * commands against a partially initialized interface. + */ +static int se_if_misc_register(struct se_if_priv *priv) +{ + int ret; + + ret = misc_register(priv->priv_dev_ctx->miscdev); + if (ret) + return dev_err_probe(priv->dev, ret, + "Failed to register misc device."); + + priv->open_gate->registered = true; + + return 0; +} + +static void se_if_priv_release(struct kref *kref) +{ + struct se_if_priv *priv = container_of(kref, struct se_if_priv, refcount); + + /* + * Free priv_dev_ctx if it still exists. se_if_priv_release() always + * runs after se_if_probe_cleanup() has completed: the initial kref + * held by probe_cleanup is the last one dropped by probe_cleanup + * itself, so no other kref_put() can reach zero -- and therefore + * trigger se_if_priv_release() -- until probe_cleanup's own + * kref_put() fires. By that time cleanup_se_shared_mem() and + * of_reserved_mem_device_release() have already run in + * probe_cleanup, so only the struct itself and its devname string + * need to be freed here. Calling cleanup_se_shared_mem() again + * would be a use-after-free of already-freed DMA memory. + */ + if (priv->priv_dev_ctx) { + kfree(priv->priv_dev_ctx->devname); + kfree(priv->priv_dev_ctx); + priv->priv_dev_ctx = NULL; + } + /* + * Be defensive: if teardown did not already drop the device-owned + * gate reference for some reason, release it here. + */ + if (priv->open_gate) { + se_if_open_gate_put(priv->open_gate); + priv->open_gate = NULL; + } + + /* + * Drop the reference on priv->dev taken in se_if_probe(). The device was + * pinned so that a file descriptor closed after device unbind can still + * safely pass priv->dev to dma_free_coherent()/dev_warn(). + */ + put_device(priv->dev); + mutex_destroy(&priv->load_fw.load_fw_lock); + mutex_destroy(&priv->modify_lock); + mutex_destroy(&priv->se_if_cmd_lock); + mutex_destroy(&priv->msg_excl_flow.msg_excl_flow_lock); + + /* Free any remaining resources that weren't devm-managed */ + kfree(priv); +} + +static void se_if_dev_ctx_release(struct kref *kref) +{ + struct se_if_device_ctx *dev_ctx = + container_of(kref, struct se_if_device_ctx, refcount); + struct se_if_priv *priv = dev_ctx->priv; + + kfree(dev_ctx); + + /* drop the priv reference owned by this device context */ + kref_put(&priv->refcount, se_if_priv_release); +} + +/* + * se_reserve_msg_if() - reserve the SE messaging interface for the current task. + * + * Blocks on msg_excl_flow_lock until this task owns the reservation, then + * publishes current as msg_excl_owner under msg_excl_lock. While a reservation + * is held, ele_msg_send_rcv() lets only the owning task issue transactions and + * rejects every other caller with -EBUSY. Used by the fw_busy recovery flow in + * se_clear_fw_busy() to drive its teardown-close messages through the otherwise + * closed circuit breaker. + * + * If a second flow tries to reserve while the interface is already reserved, + * it sleeps on msg_excl_flow_lock until the current owner calls + * se_release_msg_if(). Must be called from process/workqueue context (it may + * sleep) and every successful call must be balanced by se_release_msg_if(). + * + * Return: 0 (the reservation is always acquired once this returns). + */ +int se_reserve_msg_if(struct se_if_priv *priv) +{ + unsigned long flags; + + mutex_lock(&priv->msg_excl_flow.msg_excl_flow_lock); + /* + * The mutex guarantees this task is now the sole reserver, so + * msg_excl_owner is either NULL or already current. Publish current + * under msg_excl_lock so the lockless READ_ONCE in ele_msg_send_rcv() + * observes a consistent pointer. + */ + spin_lock_irqsave(&priv->msg_excl_flow.msg_excl_lock, flags); + priv->msg_excl_flow.msg_excl_owner = current; + spin_unlock_irqrestore(&priv->msg_excl_flow.msg_excl_lock, flags); + + return 0; +} + +/* se_release_msg_if() - release a reservation taken by se_reserve_msg_if(). */ +void se_release_msg_if(struct se_if_priv *priv) +{ + unsigned long flags; + + spin_lock_irqsave(&priv->msg_excl_flow.msg_excl_lock, flags); + priv->msg_excl_flow.msg_excl_owner = NULL; + spin_unlock_irqrestore(&priv->msg_excl_flow.msg_excl_lock, flags); + mutex_unlock(&priv->msg_excl_flow.msg_excl_flow_lock); +} + +/* se_clear_fw_busy() - atomically clear fw_busy and reclaim the parked dev_ctx. */ +static void se_clear_fw_busy(struct se_if_priv *priv) +{ + struct fw_busy_info *fbusy_info = &priv->fw_busy_info; + struct se_if_device_ctx *dev_ctx = NULL; + unsigned long flags; + + scoped_guard(spinlock_irqsave, &fbusy_info->fw_busy_lock) { + dev_ctx = fbusy_info->fw_busy_dev_ctx; + fbusy_info->fw_busy_dev_ctx = NULL; + + if (!dev_ctx) { + /* + * No parked context: nothing to recover. Clear fw_busy + * and return without reserving the interface, so the + * no-op path never leaves a dangling recovery + * reservation. The scoped_guard releases fw_busy_lock + * on this return. + */ + atomic_set(&fbusy_info->fw_busy, 0); + return; + } + } + + /* + * A context is parked and its handles must be recovered. Keep + * fw_busy set (breaker stays closed to all third parties) and + * reserve the SE interface exclusively for this recovery flow by + * publishing the current task as msg_excl_owner. ele_msg_send_rcv() + * then lets only this task's teardown-close messages through and + * rejects everyone else with -EBUSY. The reservation is assigned here, + * from outside ele_msg_send_rcv(), and released with se_release_msg_if() + * at the end, after which the interface is available for general + * se_if_cmd_lock message exchange. + * + * The scoped_guard above has already dropped fw_busy_lock before this + * se_reserve_msg_if() call: the reserve helper takes msg_excl_lock, and + * taking it while still holding fw_busy_lock would introduce a new + * fw_busy_lock -> msg_excl_lock nesting. The brief fw_busy == 1 / + * owner == NULL window that this opens is harmless - the breaker is + * fully closed, so every caller (including a would-be re-arm) is + * rejected with -EBUSY. + */ + se_reserve_msg_if(priv); + + scoped_guard(mutex, &dev_ctx->fops_lock) { + /* + * Snapshot any orphaned late FW response. On the teardown + * path se_if_probe_cleanup calls se_clear_fw_busy before + * cancel_work_sync, so fw_busy is still 1 here and a late + * IRQ can write orphan_fw_rx_msg concurrently - take + * clbk_rx_lock. On the workqueue path the IRQ writer has + * already finished; the lock is a no-contention formality. + * Call fw_api_specific_ops() outside the spinlock since it + * may sleep. + */ + u8 late_rx_snap[MAX_ALLOWED_RX_MSG_SZ]; + bool have_snap; + + spin_lock_irqsave(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); + have_snap = fbusy_info->orphan_fw_rx_msg[0] != 0; + if (have_snap) { + memcpy(late_rx_snap, fbusy_info->orphan_fw_rx_msg, + sizeof(late_rx_snap)); + memset(fbusy_info->orphan_fw_rx_msg, 0, + sizeof(fbusy_info->orphan_fw_rx_msg)); + } + spin_unlock_irqrestore(&priv->waiting_rsp_clbk_hdl.clbk_rx_lock, flags); + + if (have_snap) { + /* + * FW responded late. DMA staging buffer is no longer + * being written - safe to reclaim. Close any firmware + * resource handle carried in the response. + */ + fw_api_specific_ops(priv->priv_dev_ctx, + (struct se_api_msg *)late_rx_snap, true); + + if (dev_ctx == priv->priv_dev_ctx) { + /* + * Internal context: probe-time static DMA buf; + * se_if_probe_cleanup reclaims it explicitly. + * Just reset logical pos and return gen_pool + * loan buffers for reuse. + */ + if (priv->mem_pool) + se_cleanup_mem_pool_buf(dev_ctx, true); + se_dev_ctx_shared_mem_cleanup(dev_ctx); + } else if (dev_ctx->cleanup_done) { + /* + * Userspace fd already closed while fw_busy was + * armed (e.g. SIGKILL). FW has now responded; + * close deferred handles and free the DMA buf. + */ + if (dev_ctx->strg_hdl && + se_close_storage(priv->priv_dev_ctx, + dev_ctx->strg_hdl)) + dev_err(priv->dev, + "%s: failed to close deferred storage handle\n", + dev_ctx->devname); + if (dev_ctx->sess_hdl && + se_close_session(priv->priv_dev_ctx, + dev_ctx->sess_hdl)) + dev_err(priv->dev, + "%s: failed to close deferred session handle\n", + dev_ctx->devname); + dev_ctx->strg_hdl = 0; + dev_ctx->sess_hdl = 0; + cleanup_se_shared_mem(dev_ctx, true); + } else { + /* Pure timeout, fd still open: reset pos only. */ + se_dev_ctx_shared_mem_cleanup(dev_ctx); + } + } else { + /* + * have_snap=false only on teardown (FW never responded, + * or teardown beat the late IRQ, which going_away then + * drops). priv_dev_ctx is handled by probe_cleanup's + * single cleanup_se_shared_mem(reclaim=true) after + * cancel_work_sync, so nothing to do here. For a + * userspace dev_ctx (cleanup_done already true), use + * reclaim=false: the pos gate leaks the buffer if FW may + * still be writing, else frees it. reclaim=true would + * also be safe here since the region is no-map. + */ + if (dev_ctx != priv->priv_dev_ctx && dev_ctx->cleanup_done) + cleanup_se_shared_mem(dev_ctx, false); + } + } + + /* + * Recovery flow is done: release the exclusive reservation, then clear + * the breaker. se_release_msg_if() drops msg_excl_owner; the fw_busy + * clear below reopens the interface. Ordering is safe either way: while + * fw_busy is still 1 a third party is rejected regardless of owner, and + * once fw_busy is 0 the owner is no longer consulted. + */ + se_release_msg_if(priv); + + spin_lock_irqsave(&fbusy_info->fw_busy_lock, flags); + atomic_set(&fbusy_info->fw_busy, 0); + spin_unlock_irqrestore(&fbusy_info->fw_busy_lock, flags); + kref_put(&dev_ctx->refcount, se_if_dev_ctx_release); +} + +void unset_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx) +{ + struct se_if_priv *priv = dev_ctx->priv; + struct se_api_msg *old_rx_msg = NULL; + struct se_clbk_handle *se_clbk_hdl; + unsigned long flags; + + lockdep_assert_held(&priv->modify_lock); + + se_clbk_hdl = &priv->cmd_receiver_clbk_hdl; + + if (se_clbk_hdl->dev_ctx == dev_ctx) { + spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); + old_rx_msg = se_clbk_hdl->rx_msg; + se_clbk_hdl->dev_ctx = NULL; + se_clbk_hdl->rx_msg = NULL; + se_clbk_hdl->rx_msg_sz = 0; + spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + + kfree(old_rx_msg); + complete_all(&se_clbk_hdl->done); + } +} + +/* + * check_cmd_rcvr_status() - check whether dev_ctx can become the command + * receiver or is already become the command receiver. + * + * Returns: + * 0 - dev_ctx is already the registered receiver + * -EBUSY - another context is already the receiver + * -EINVAL - dev_ctx has no storage handle + * -ENXIO - ready to proceed: no receiver set, strg_hdl present + * + * Caller must hold priv->modify_lock. + */ +static int check_cmd_rcvr_status(struct se_if_device_ctx *dev_ctx) +{ + struct se_if_priv *priv = dev_ctx->priv; + struct se_clbk_handle *se_clbk_hdl = &priv->cmd_receiver_clbk_hdl; + + lockdep_assert_held(&priv->modify_lock); + + if (se_clbk_hdl->dev_ctx == dev_ctx) + return 0; + + if (se_clbk_hdl->dev_ctx) + return -EBUSY; + + if (!dev_ctx->strg_hdl) + return -EINVAL; + + /* Reaching here means, with a valid storage handle and command-receiver as NULL, + * either the registration process is to be done or failed. + */ + return -ENXIO; +} + +int set_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx) +{ + struct se_if_priv *priv = dev_ctx->priv; + struct se_clbk_handle *se_clbk_hdl = &priv->cmd_receiver_clbk_hdl; + struct se_api_msg *new_rx_msg = NULL; + unsigned long flags; + int ret; + + guard(mutex)(&priv->modify_lock); + + /* + * All state checks happen inside modify_lock so the result cannot + * go stale between the check and the arming below. + */ + ret = check_cmd_rcvr_status(dev_ctx); + if (ret != -ENXIO) + return ret; + + if (!se_clbk_hdl->rx_msg) { + new_rx_msg = kzalloc(MAX_NVM_MSG_LEN, GFP_KERNEL); + if (!new_rx_msg) + return -ENOMEM; + } + spin_lock_irqsave(&se_clbk_hdl->clbk_rx_lock, flags); + if (new_rx_msg) + se_clbk_hdl->rx_msg = new_rx_msg; + reinit_completion(&se_clbk_hdl->done); + se_clbk_hdl->rx_msg_sz = MAX_NVM_MSG_LEN; + se_clbk_hdl->dev_ctx = dev_ctx; + dev_ctx->rcv_msg_timeout_jiffies = MAX_SCHEDULE_TIMEOUT; + spin_unlock_irqrestore(&se_clbk_hdl->clbk_rx_lock, flags); + + return 0; +} + +static void dlink_dev_ctx(struct se_if_device_ctx *dev_ctx) +{ + struct se_if_priv *priv = dev_ctx->priv; + + unset_dev_ctx_as_command_receiver(dev_ctx); + + if (!list_empty(&dev_ctx->link)) { + list_del_init(&dev_ctx->link); + priv->active_devctx_count--; + } +} + +bool se_is_fw_busy_ctx(struct se_if_device_ctx *dev_ctx) +{ + struct se_if_priv *priv = dev_ctx->priv; + struct fw_busy_info *fbusy_info = &priv->fw_busy_info; + unsigned long flags; + bool match; + + spin_lock_irqsave(&fbusy_info->fw_busy_lock, flags); + match = fbusy_info->fw_busy_dev_ctx == dev_ctx; + spin_unlock_irqrestore(&fbusy_info->fw_busy_lock, flags); + + return match; +} + +static void cleanup_dev_ctx(struct se_if_device_ctx *dev_ctx, bool is_fclose) +{ + struct fw_busy_info *fbusy_info = &dev_ctx->priv->fw_busy_info; + bool already_done; + + scoped_guard(mutex, &dev_ctx->fops_lock) { + already_done = dev_ctx->cleanup_done; + if (!already_done) { + /* + * Ask FW to drop this context's session and storage so + * the kernel and FW stay in sync. Done here, under this + * context's fops_lock only (not the global modify_lock), + * because both close requests block on a firmware + * round-trip; issuing them while modify_lock was held + * would stall every other context for the FW timeout. + * + * Skip the round-trips once the FW path is marked busy. + * fw_busy is armed when a synchronous transaction times + * out; while it is set ele_msg_send_rcv() rejects further + * commands with -EBUSY without waiting. It is only cleared + * by se_clear_fw_busy(), which during unbind runs once + * after this loop (or earlier from fw_busy_work only if a + * genuine late FW response arrives). On a hung FW no late + * response comes, so the breaker stays set for the rest of + * the loop and the remaining closes would just return + * -EBUSY and log spurious "failed to close" errors. Skip + * them and emit a single warning instead. + */ + if (atomic_read(&fbusy_info->fw_busy)) { + if (dev_ctx->strg_hdl || dev_ctx->sess_hdl) + dev_warn(dev_ctx->priv->dev, + "%s: skipping session/storage close, FW is busy\n", + dev_ctx->devname); + } else { + /* + * Choose which dev_ctx sends the close messages. + * fclose: use the caller's own dev_ctx so a race with + * unbind is rejected with -ENODEV instead of hitting a + * freed tx_chan. Teardown: use priv_dev_ctx; going_away + * is set but the reservation (msg_excl_owner == current) + * lets these closes through while tx_chan is still live. + */ + struct se_if_device_ctx *tx_ctx = is_fclose ? dev_ctx : + dev_ctx->priv->priv_dev_ctx; + + if (dev_ctx->strg_hdl && + se_close_storage(tx_ctx, dev_ctx->strg_hdl)) + dev_err(dev_ctx->priv->dev, "failed to close storage.\n"); + if (dev_ctx->sess_hdl && + se_close_session(tx_ctx, dev_ctx->sess_hdl)) + dev_err(dev_ctx->priv->dev, "failed to close session.\n"); + } + /* + * fw_busy is caused by one timed-out synchronous transaction. + * Only that transaction's dev_ctx may still have coherent + * memory referenced by FW. Do not skip cleanup for unrelated + * contexts while fw_busy is set. + */ + if (se_is_fw_busy_ctx(dev_ctx)) + dev_warn(dev_ctx->priv->dev, + "%s: deferring shared memory cleanup while FW is busy\n", + dev_ctx->devname); + else + cleanup_se_shared_mem(dev_ctx, true); + + kfree(dev_ctx->devname); + dev_ctx->devname = NULL; + dev_ctx->cleanup_done = true; + } + } + + if (is_fclose) + kref_put(&dev_ctx->refcount, se_if_dev_ctx_release); +} + +static void dlink_n_cleanup_dev_ctx(struct se_if_device_ctx *dev_ctx, bool is_fclose) +{ + struct se_if_priv *priv = dev_ctx->priv; + + if (is_fclose) { + scoped_guard(mutex, &priv->modify_lock) + dlink_dev_ctx(dev_ctx); + } + + cleanup_dev_ctx(dev_ctx, is_fclose); +} + +static int init_device_context(struct se_if_priv *priv, int ch_id, + struct se_if_device_ctx **new_dev_ctx) +{ + struct se_if_device_ctx *dev_ctx; + int ret = 0; + + dev_ctx = kzalloc_obj(*dev_ctx); + + if (!dev_ctx) + return -ENOMEM; + + dev_ctx->devname = kasprintf(GFP_KERNEL, "%s0_ch%d", + get_se_if_name(priv->if_defs->se_if_type), + ch_id); + if (!dev_ctx->devname) { + kfree(dev_ctx); + return -ENOMEM; + } + + mutex_init(&dev_ctx->fops_lock); + kref_init(&dev_ctx->refcount); + dev_ctx->priv = priv; + dev_ctx->cleanup_done = false; + INIT_LIST_HEAD(&dev_ctx->link); + set_se_rcv_msg_timeout(dev_ctx, SE_RCV_MSG_LONG_TIMEOUT_MS); + *new_dev_ctx = dev_ctx; + + ret = init_se_shared_mem(dev_ctx); + if (ret < 0) { + kfree(dev_ctx->devname); + kfree(dev_ctx); + *new_dev_ctx = NULL; + + return ret; + } + + /* Take a reference to priv for this device context */ + kref_get(&priv->refcount); + + scoped_guard(mutex, &priv->modify_lock) { + list_add_tail(&dev_ctx->link, &priv->dev_ctx_list); + priv->active_devctx_count++; + } + + return ret; +} + +static int se_ioctl_cmd_snd_rcv_cleanup(struct se_if_device_ctx *dev_ctx, void __user *uarg, + struct se_ioctl_cmd_snd_rcv_rsp_info *cmd_snd_rcv_rsp_info) +{ + /* shared memory is allocated before this IOCTL */ + se_dev_ctx_shared_mem_cleanup(dev_ctx); + + if (cmd_snd_rcv_rsp_info->rx_buf_sz && + copy_to_user(uarg, cmd_snd_rcv_rsp_info, sizeof(*cmd_snd_rcv_rsp_info))) { + dev_err(dev_ctx->priv->dev, "%s: Failed to copy cmd_snd_rcv_rsp_info to user.\n", + dev_ctx->devname); + return -EFAULT; + } + + return 0; +} + +static int se_ioctl_cmd_snd_rcv_rsp_handler(struct se_if_device_ctx *dev_ctx, + void __user *uarg) +{ + struct se_ioctl_cmd_snd_rcv_rsp_info cmd_snd_rcv_rsp_info = {0}; + struct se_if_priv *priv = dev_ctx->priv; + int rsp_status_err = 0; + int act_rx_msg_sz = 0; + int cleanup_err = 0; + int err = 0; + + if (copy_from_user(&cmd_snd_rcv_rsp_info, uarg, + sizeof(cmd_snd_rcv_rsp_info))) { + dev_err(priv->dev, + "%s: Failed to copy cmd_snd_rcv_rsp_info from user.", + dev_ctx->devname); + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return -EFAULT; + } + + if (cmd_snd_rcv_rsp_info.tx_buf_sz < SE_MU_HDR_SZ || + cmd_snd_rcv_rsp_info.tx_buf_sz > MAX_ALLOWED_TX_MSG_SZ) { + dev_err(priv->dev, "%s: User buffer too small/large(%d < %d)\n", + dev_ctx->devname, cmd_snd_rcv_rsp_info.tx_buf_sz, + cmd_snd_rcv_rsp_info.tx_buf_sz < SE_MU_HDR_SZ ? SE_MU_HDR_SZ : + MAX_ALLOWED_TX_MSG_SZ); + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return -ENOSPC; + } + + struct se_api_msg *tx_msg __free(kfree) = + memdup_user(u64_to_user_ptr(cmd_snd_rcv_rsp_info.tx_buf), + cmd_snd_rcv_rsp_info.tx_buf_sz); + if (IS_ERR(tx_msg)) { + err = PTR_ERR(tx_msg); + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return err; + } + + err = se_chk_tx_cmd_msg_hdr(dev_ctx, &tx_msg->header, + cmd_snd_rcv_rsp_info.tx_buf_sz, + cmd_snd_rcv_rsp_info.rx_buf_sz); + if (err) { + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return err; + } + + if (cmd_snd_rcv_rsp_info.rx_buf_sz < SE_MU_HDR_SZ || + cmd_snd_rcv_rsp_info.rx_buf_sz > MAX_ALLOWED_RX_MSG_SZ) { + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return -EINVAL; + } + + if (tx_msg->header.tag != priv->if_defs->cmd_tag) { + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return -EINVAL; + } + + if (tx_msg->header.ver == priv->if_defs->fw_api_ver && + get_load_fw_instance(priv)->is_fw_tobe_loaded) { + err = se_load_firmware(priv); + if (err) { + dev_err(priv->dev, "Could not send msg as FW is not loaded.\n"); + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return -EPERM; + } + } + + struct se_api_msg *rx_msg __free(kfree) = + kzalloc(cmd_snd_rcv_rsp_info.rx_buf_sz, GFP_KERNEL); + if (!rx_msg) { + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return -ENOMEM; + } + + err = ele_msg_send_rcv(dev_ctx, tx_msg, cmd_snd_rcv_rsp_info.tx_buf_sz, + rx_msg, cmd_snd_rcv_rsp_info.rx_buf_sz, &act_rx_msg_sz); + if (err < 0) { + /* + * -ERESTARTSYS here means the wait was interrupted by a signal + * after the command had already been handed to - and executed + * by - the firmware, with its response delivered into rx_msg + * (ele_msg_send_rcv() converts only a positive, i.e. successfully + * received, result to -ERESTARTSYS). If that response carried a + * freshly allocated session/storage handle, record it now via + * fw_api_specific_ops(): the handle is already live in firmware, + * so leaving it untracked would stop cleanup_dev_ctx() from ever + * closing it and leak the firmware resource. Validate the + * delivered response first, using its own declared length bounded + * by the caller's buffer, so a truncated or malformed reply is + * not acted upon. + */ + if (err == -ERESTARTSYS) { + u32 rsp_sz = rx_msg->header.size << 2; + + if (rsp_sz && rsp_sz <= cmd_snd_rcv_rsp_info.rx_buf_sz && + !se_val_rsp_hdr_n_status(dev_ctx, rx_msg, + tx_msg->header.command, act_rx_msg_sz, + tx_msg->header.ver)) { + se_dev_ctx_cpy_out_data(dev_ctx); + fw_api_specific_ops(dev_ctx, rx_msg, true); + } + /* + * NOTE: se_dev_ctx_cpy_out_data() above has already + * copied the firmware response payload to userspace before + * this point. Returning -EINTR here is intentional, not + * -ERESTARTSYS: the VFS would transparently restart the + * ioctl on -ERESTARTSYS, re-issuing the command with + * already-zeroed shared input buffers. -EINTR prevents + * auto-restart and lets userspace enter its signal handler + * to decide whether to reissue the command. + * See Documentation/driver-api/firmware/other_interfaces.rst, + * section "Signal handling after a completed hardware + * operation". + */ + err = -EINTR; + } + + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + + return err; + } + + /* + * ele_msg_send_rcv() returns a positive received-message size on + * success. Returning that raw size as the ioctl result would make a + * successful transaction look like a positive (non-zero) return value + * to userspace. Record the actual received size in rx_buf_sz for the + * response copied back to userspace, then normalise err to 0 so the + * ioctl reports plain success; the firmware status is conveyed to + * userspace inside the response buffer itself. + */ + cmd_snd_rcv_rsp_info.rx_buf_sz = act_rx_msg_sz; + err = 0; + + dev_dbg(priv->dev, "%s: %s %s.\n", dev_ctx->devname, __func__, + "message received, start transmit to user"); + + rsp_status_err = + se_val_rsp_hdr_n_status(dev_ctx, rx_msg, tx_msg->header.command, + act_rx_msg_sz, tx_msg->header.ver); + + if (!rsp_status_err) { + /* + * For msg IDs handled by fw_api_specific_ops(), the exact + * response size was already ensured in ele_uapi_allowed_fw_cmd(). + */ + err = fw_api_specific_ops(dev_ctx, rx_msg, false); + if (err) { + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return err; + } + + err = se_dev_ctx_cpy_out_data(dev_ctx); + if (err < 0) { + se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + return err; + } + } + + /* Copy data from the buffer */ + print_hex_dump_debug("to user ", DUMP_PREFIX_OFFSET, 4, 4, rx_msg, + cmd_snd_rcv_rsp_info.rx_buf_sz, false); + + if (copy_to_user(u64_to_user_ptr(cmd_snd_rcv_rsp_info.rx_buf), rx_msg, + cmd_snd_rcv_rsp_info.rx_buf_sz)) { + dev_err(priv->dev, "%s: Failed to copy to user.\n", dev_ctx->devname); + err = -EFAULT; + } + + cleanup_err = se_ioctl_cmd_snd_rcv_cleanup(dev_ctx, uarg, &cmd_snd_rcv_rsp_info); + + if (cleanup_err && !err) + err = cleanup_err; + + return err; +} + +static int se_ioctl_get_mu_info(struct se_if_device_ctx *dev_ctx, + void __user *uarg) +{ + struct se_if_priv *priv = dev_ctx->priv; + struct se_ioctl_get_if_info if_info; + struct se_if_node *if_node; + int err = 0; + + if_node = container_of(priv->if_defs, typeof(*if_node), if_defs); + + if_info.se_if_id = 0; + if_info.interrupt_idx = 0; + if_info.tz = 0; + if_info.did = 0; + if_info.cmd_tag = priv->if_defs->cmd_tag; + if_info.rsp_tag = priv->if_defs->rsp_tag; + if_info.success_tag = priv->if_defs->success_tag; + if_info.base_api_ver = priv->if_defs->base_api_ver; + if_info.fw_api_ver = priv->if_defs->fw_api_ver; + + dev_dbg(priv->dev, "%s: info [se_if_id: %d, irq_idx: %d, tz: 0x%x, did: 0x%x].\n", + dev_ctx->devname, if_info.se_if_id, if_info.interrupt_idx, if_info.tz, + if_info.did); + + if (copy_to_user(uarg, &if_info, sizeof(if_info))) { + dev_err(priv->dev, "%s: Failed to copy mu info to user.\n", + dev_ctx->devname); + err = -EFAULT; + } + + return err; +} + +static void rollback_shared_mem_pos(struct se_if_device_ctx *dev_ctx, u32 length) +{ + struct se_shared_mem *shared_mem = NULL; + + shared_mem = &dev_ctx->se_shared_mem_mgmt.non_secure_mem; + + if (WARN_ON_ONCE(length > shared_mem->pos)) { + shared_mem->pos = 0; + return; + } + + shared_mem->pos -= length; +} + +int get_shared_mem_slot(struct se_if_device_ctx *dev_ctx, + u32 *length, dma_addr_t *ele_dma_addr, void **ptr) +{ + struct se_shared_mem *shared_mem = NULL; + bool is_fw_busy_dev_ctx; + size_t aligned_len = 0; + u32 pos; + + /* + * If this context is the one that caused a firmware timeout the shared + * DMA buffers may still be actively read/written by the firmware. + */ + is_fw_busy_dev_ctx = se_is_fw_busy_ctx(dev_ctx); + if (is_fw_busy_dev_ctx) + return -EBUSY; + + aligned_len = round_up((size_t)*length, 8); + if (aligned_len < *length) { + dev_err(dev_ctx->priv->dev, "%s: Invalid buffer length.\n", + dev_ctx->devname); + return -EINVAL; + } + + /* No specific requirement for this buffer. */ + shared_mem = &dev_ctx->se_shared_mem_mgmt.non_secure_mem; + + /* Check there is enough space in the shared memory. */ + dev_dbg(dev_ctx->priv->dev, "%s: req_size = %zd, max_size= %d, curr_pos = %d\n", + dev_ctx->devname, aligned_len, shared_mem->size, + shared_mem->pos); + + if (shared_mem->size < shared_mem->pos || + aligned_len > (shared_mem->size - shared_mem->pos)) { + dev_err(dev_ctx->priv->dev, "%s: Not enough space in shared memory.\n", + dev_ctx->devname); + return -ENOMEM; + } + + /* Allocate space in shared memory. 8 bytes aligned. */ + pos = shared_mem->pos; + shared_mem->pos += aligned_len; + *ele_dma_addr = (u64)shared_mem->dma_addr + pos; + *ptr = shared_mem->ptr + pos; + *length = aligned_len; + + memset(shared_mem->ptr + pos, 0, aligned_len); + + return 0; +} + +/* + * Copy a buffer of data to/from the user and return the address to use in + * messages + */ +static int se_ioctl_setup_iobuf_handler(struct se_if_device_ctx *dev_ctx, + void __user *uarg) +{ + struct se_ioctl_setup_iobuf io = {0}; + struct se_buf_desc *b_desc = NULL; + void *dma_buf_ptr = NULL; + dma_addr_t ele_dma_addr; + u32 aligned_len = 0; + int err = 0; + + if (copy_from_user(&io, uarg, sizeof(io))) { + dev_err(dev_ctx->priv->dev, "%s: Failed copy iobuf config from user.\n", + dev_ctx->devname); + return -EFAULT; + } + + dev_dbg(dev_ctx->priv->dev, "%s: io [buf: %p(%d) flag: %x].\n", dev_ctx->devname, + u64_to_user_ptr(io.user_buf), io.length, io.flags); + + if (io.length == 0 || !io.user_buf) { + /* + * Accept NULL pointers since some buffers are optional + * in FW commands. In this case we should return 0 as + * pointer to be embedded into the message. + * Skip all data copy part of code below. + */ + io.ele_addr = 0; + goto copy; + } + + aligned_len = io.length; + err = get_shared_mem_slot(dev_ctx, &aligned_len, &ele_dma_addr, &dma_buf_ptr); + if (err) + return err; + + io.ele_addr = ele_dma_addr; + if ((io.flags & SE_IO_BUF_FLAGS_IS_INPUT) || + (io.flags & SE_IO_BUF_FLAGS_IS_IN_OUT)) { + /* + * buffer is input: + * copy data from user space to this allocated buffer. + */ + if (copy_from_user(dma_buf_ptr, u64_to_user_ptr(io.user_buf), + io.length)) { + dev_err(dev_ctx->priv->dev, + "%s: Failed copy data to shared memory.", + dev_ctx->devname); + err = -EFAULT; + goto rollback; + } + } + + b_desc = add_b_desc_to_pending_list(dma_buf_ptr, &io, dev_ctx); + if (IS_ERR(b_desc)) { + err = PTR_ERR(b_desc); + dev_err(dev_ctx->priv->dev, "%s: Failed to allocate/link b_desc.\n", + dev_ctx->devname); + goto rollback; + } + +copy: + /* Provide the EdgeLock Enclave address to user space only if success.*/ + if (copy_to_user(uarg, &io, sizeof(io))) { + dev_err(dev_ctx->priv->dev, "%s: Failed to copy iobuff setup to user.\n", + dev_ctx->devname); + err = -EFAULT; + goto rollback; + } + return err; + +rollback: + if (!IS_ERR_OR_NULL(b_desc)) { + list_del(&b_desc->link); + kfree(b_desc); + } + + if (dma_buf_ptr && aligned_len) { + memset(dma_buf_ptr, 0, aligned_len); + rollback_shared_mem_pos(dev_ctx, aligned_len); + } + + return err; +} + +/* IOCTL to provide SoC information */ +static int se_ioctl_get_se_soc_info_handler(struct se_if_device_ctx *dev_ctx, + void __user *uarg) +{ + struct se_ioctl_get_soc_info soc_info; + int err = -EINVAL; + + soc_info.soc_id = get_se_soc_id(dev_ctx->priv); + soc_info.soc_rev = var_se_info.soc_rev; + + err = copy_to_user(uarg, (u8 *)(&soc_info), sizeof(soc_info)); + if (err) { + dev_err(dev_ctx->priv->dev, "%s: Failed to copy soc info to user.\n", + dev_ctx->devname); + err = -EFAULT; + } + + return err; +} + +/* + * File operations for user-space + */ + +/* Write a message to the MU. */ +static ssize_t se_if_fops_write(struct file *fp, const char __user *buf, + size_t size, loff_t *ppos) +{ + struct se_if_device_ctx *dev_ctx = fp->private_data; + struct se_if_priv *priv; + int err; + + scoped_cond_guard(mutex_intr, return -ERESTARTSYS, &dev_ctx->fops_lock) { + if (dev_ctx->cleanup_done) + return -ENODEV; + + priv = dev_ctx->priv; + + dev_dbg(priv->dev, "%s: write from buf (%p)%zu, ppos=%lld.\n", dev_ctx->devname, + buf, size, ((ppos) ? *ppos : 0)); + + if (dev_ctx != priv->cmd_receiver_clbk_hdl.dev_ctx) { + se_dev_ctx_shared_mem_cleanup(dev_ctx); + return -EINVAL; + } + + if (size < SE_MU_HDR_SZ || size > MAX_ALLOWED_TX_MSG_SZ) { + dev_err(priv->dev, "%s: User buffer too small/large(%zu < %d)\n", + dev_ctx->devname, size, + size < SE_MU_HDR_SZ ? SE_MU_HDR_SZ : + MAX_ALLOWED_TX_MSG_SZ); + return -ENOSPC; + } + + struct se_api_msg *tx_msg __free(kfree) = memdup_user(buf, size); + if (IS_ERR(tx_msg)) + return PTR_ERR(tx_msg); + + err = se_chk_tx_rsp_msg_hdr(dev_ctx, &tx_msg->header, size); + if (err) + return err; + + print_hex_dump_debug("from user ", DUMP_PREFIX_OFFSET, 4, 4, + tx_msg, size, false); + + err = ele_msg_send(dev_ctx, tx_msg, size); + + return err; + } +} + +/* + * Read a message from the MU. + * Blocking until a message is available. + */ +static ssize_t se_if_fops_read(struct file *fp, char __user *buf, size_t size, + loff_t *ppos) +{ + struct se_if_device_ctx *dev_ctx = fp->private_data; + u8 rx_msg_snap[MAX_NVM_MSG_LEN] = {}; + char devname_snap[32] = {}; + struct se_if_priv *priv; + unsigned long flags; + size_t copy_len; + int err; + + scoped_cond_guard(mutex_intr, return -ERESTARTSYS, &dev_ctx->fops_lock) { + priv = dev_ctx->priv; + + if (dev_ctx->cleanup_done) + return -ENODEV; + + /* + * Snapshot devname once while fops_lock is held. After the + * scoped guard releases the lock, a concurrent cleanup_dev_ctx() + * could free dev_ctx->devname before the error path below runs. + */ + strscpy(devname_snap, dev_ctx->devname, sizeof(devname_snap)); + + dev_dbg(priv->dev, "%s: read to buf %p(%zu), ppos=%lld.\n", devname_snap, + buf, size, ((ppos) ? *ppos : 0)); + + mutex_lock(&priv->modify_lock); + if (dev_ctx != priv->cmd_receiver_clbk_hdl.dev_ctx) { + mutex_unlock(&priv->modify_lock); + se_dev_ctx_shared_mem_cleanup(dev_ctx); + return -EINVAL; + } + mutex_unlock(&priv->modify_lock); + } + + err = ele_msg_rcv(dev_ctx, &priv->cmd_receiver_clbk_hdl); + if (err < 0) { + if (err != -ERESTARTSYS) + dev_err(priv->dev, + "%s: Er[0x%x]: Signal Interrupted. Current act-dev-ctx count: %d.", + devname_snap, err, dev_ctx->priv->active_devctx_count); + return err; + } + + /* + * Reacquire fops_lock before touching any dev_ctx state (pending lists, + * rx_msg) after the blocking wait. fops_lock was dropped before calling + * ele_msg_rcv(). If cleanup_dev_ctx() ran concurrently it could have + * freed the DMA buffers and the pending lists, leading to UAF and list + * corruption. Re-checking cleanup_done under fops_lock prevents that. + */ + mutex_lock(&dev_ctx->fops_lock); + + if (dev_ctx->cleanup_done) { + mutex_unlock(&dev_ctx->fops_lock); + return -ENODEV; + } + + /* + * Snapshot the whole rx_msg under modify_lock + clbk_rx_lock, not just + * copy_len bytes: fw_api_specific_ops() reads data words (e.g. strg_hdl + * at data[1]) beyond the userspace read size; truncating would record a + * zero handle. Run fw_api_specific_ops() OUTSIDE modify_lock + * (ELE_STORAGE_OPEN_REQ re-takes it, else deadlock). + */ + scoped_guard(mutex, &priv->modify_lock) { + spin_lock_irqsave(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags); + if (priv->cmd_receiver_clbk_hdl.dev_ctx != dev_ctx || + !priv->cmd_receiver_clbk_hdl.rx_msg || + !priv->cmd_receiver_clbk_hdl.rx_msg_sz) { + spin_unlock_irqrestore(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags); + mutex_unlock(&dev_ctx->fops_lock); + return -ENODEV; + } + copy_len = min(size, (size_t)priv->cmd_receiver_clbk_hdl.rx_msg_sz); + memcpy(rx_msg_snap, priv->cmd_receiver_clbk_hdl.rx_msg, + priv->cmd_receiver_clbk_hdl.rx_msg_sz); + priv->cmd_receiver_clbk_hdl.rx_msg_sz = 0; + spin_unlock_irqrestore(&priv->cmd_receiver_clbk_hdl.clbk_rx_lock, flags); + + /* We may need to copy the output data to user before + * delivering the completion message. + */ + err = se_dev_ctx_cpy_out_data(dev_ctx); + if (err < 0) { + se_dev_ctx_shared_mem_cleanup(dev_ctx); + mutex_unlock(&dev_ctx->fops_lock); + return err; + } + } + + /* fw_api_specific_ops() runs outside modify_lock; see comment above. */ + print_hex_dump_debug("to user ", DUMP_PREFIX_OFFSET, 4, 4, + rx_msg_snap, copy_len, false); + + cmd_receiver_specific_ops(dev_ctx, (struct se_api_msg *)rx_msg_snap); + err = copy_len; + if (copy_to_user(buf, rx_msg_snap, copy_len)) + err = -EFAULT; + + se_dev_ctx_shared_mem_cleanup(dev_ctx); + mutex_unlock(&dev_ctx->fops_lock); + + return err; +} + +/* Open a character device. */ +static int se_if_fops_open(struct inode *nd, struct file *fp) +{ + struct miscdevice *miscdev = fp->private_data; + struct se_if_open_gate *gate; + struct se_if_device_ctx *misc_dev_ctx; + struct se_if_device_ctx *dev_ctx; + struct se_if_priv *priv; + int err = 0; + + gate = container_of(miscdev, struct se_if_open_gate, miscdev); + + if (!se_if_open_gate_get(gate)) + return -ENODEV; + + if (mutex_lock_interruptible(&gate->lock)) { + se_if_open_gate_put(gate); + return -ERESTARTSYS; + } + + if (gate->dying || !gate->priv || + !kref_get_unless_zero(&gate->priv->refcount)) { + mutex_unlock(&gate->lock); + se_if_open_gate_put(gate); + return -ENODEV; + } + + priv = gate->priv; + mutex_unlock(&gate->lock); + + misc_dev_ctx = priv->priv_dev_ctx; + + if (mutex_lock_interruptible(&misc_dev_ctx->fops_lock)) { + err = -ERESTARTSYS; + goto out_put_priv; + } + + if (misc_dev_ctx->cleanup_done) { + err = -ENODEV; + goto out_unlock_misc; + } + + priv->dev_ctx_mono_count++; + err = init_device_context(priv, priv->dev_ctx_mono_count, &dev_ctx); + if (err) { + dev_err(priv->dev, "Failed to create dev-ctx.\n"); + goto out_unlock_misc; + } + + fp->private_data = dev_ctx; + +out_unlock_misc: + mutex_unlock(&misc_dev_ctx->fops_lock); +out_put_priv: + kref_put(&priv->refcount, se_if_priv_release); + se_if_open_gate_put(gate); + return err; +} + +/* Close a character device. */ +static int se_if_fops_close(struct inode *nd, struct file *fp) +{ + struct se_if_device_ctx *dev_ctx = fp->private_data; + + dlink_n_cleanup_dev_ctx(dev_ctx, true); + + return 0; +} + +/* IOCTL entry point of a character device */ +static long se_ioctl(struct file *fp, unsigned int cmd, unsigned long arg) +{ + struct se_if_device_ctx *dev_ctx = fp->private_data; + struct se_if_priv *priv; + void __user *uarg = (void __user *)arg; + long err; + + /* Prevent race during change of device context */ + scoped_cond_guard(mutex_intr, return -ERESTARTSYS, &dev_ctx->fops_lock) { + if (dev_ctx->cleanup_done) + return -ENODEV; + + priv = dev_ctx->priv; + + switch (cmd) { + case SE_IOCTL_CHECK_CMD_RCV_REG_STATUS: { + guard(mutex)(&priv->modify_lock); + err = check_cmd_rcvr_status(dev_ctx); + break; + } + case SE_IOCTL_GET_MU_INFO: + err = se_ioctl_get_mu_info(dev_ctx, uarg); + break; + case SE_IOCTL_SETUP_IOBUF: + err = se_ioctl_setup_iobuf_handler(dev_ctx, uarg); + break; + case SE_IOCTL_GET_SOC_INFO: + err = se_ioctl_get_se_soc_info_handler(dev_ctx, uarg); + break; + case SE_IOCTL_CMD_SEND_RCV_RSP: + err = se_ioctl_cmd_snd_rcv_rsp_handler(dev_ctx, uarg); + break; + default: + err = -ENOTTY; + dev_dbg(priv->dev, "%s: IOCTL %.8x not supported.\n", + dev_ctx->devname, cmd); + } + } + + return err; +} + +/* Char driver setup */ +static const struct file_operations se_if_fops = { + .open = se_if_fops_open, + .owner = THIS_MODULE, + .release = se_if_fops_close, + .unlocked_ioctl = se_ioctl, + .compat_ioctl = compat_ptr_ioctl, + .read = se_if_fops_read, + .write = se_if_fops_write, +}; + +int se_get_mem_pool_buf(struct se_if_device_ctx *dev_ctx, void **buf, + dma_addr_t *daddr, u32 len) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct se_if_priv *priv = dev_ctx->priv; + struct se_buf_desc *b_desc = NULL; + + lockdep_assert_held(&dev_ctx->fops_lock); + + if (se_is_fw_busy_ctx(dev_ctx)) + return -EBUSY; + + b_desc = kzalloc_obj(*b_desc); + if (!b_desc) + return -ENOMEM; + + /* + * gen_pool is internally thread-safe, so contexts may allocate + * concurrently. The buffer is tracked on this context's own + * mem_pool_buf_list and released on its cleanup path. + */ + *buf = gen_pool_dma_alloc(priv->mem_pool, len, daddr); + if (!*buf) { + dev_err(priv->dev, "Failed to alloc from gen_pool.\n"); + kfree(b_desc); + return -ENOMEM; + } + + /* gen_pool_dma_alloc() does not zero the buffer. */ + memset(*buf, 0, len); + b_desc->shared_buf_ptr = *buf; + b_desc->size = len; + + list_add_tail(&b_desc->link, &se_shared_mem_mgmt->mem_pool_buf_list); + + return 0; +} + +void se_cleanup_mem_pool_buf(struct se_if_device_ctx *dev_ctx, bool reclaim) +{ + struct se_shared_mem_mgmt_info *se_shared_mem_mgmt = &dev_ctx->se_shared_mem_mgmt; + struct se_if_priv *priv = dev_ctx->priv; + struct se_buf_desc *b_desc, *temp; + + /* + * Free only the buffers this context allocated. A context that never + * used the pool has an empty list, so this is a no-op for it. + * + * Unlike the coherent staging buffer, the pool path needs no + * "nothing staged" (pos) gate on the reclaim=false leg. Pool buffers + * are ephemeral, per-transaction allocations: se_get_mem_pool_buf() + * refuses to allocate once the context is fw_busy, ele_msg_send_rcv() + * refuses to start a new command while fw_busy, and the success path + * frees the whole list via se_cleanup_mem_pool_buf(reclaim=true) + * before returning. se_if_cmd_lock serialises synchronous commands, so + * at most one transaction is outstanding. The only way to reach here + * with reclaim=false and a non-empty list is the single fw_busy + * context still owning the buffer(s) from the one timed-out + * transaction. Those buffers are exactly the in-flight ones the + * enclave may still be DMA-ing into, so leaving them on the list (no + * gen_pool_free) deliberately leaks them to avoid a DMA-after-free - + * there are no already-consumed pool buffers to reclaim on this leg. + */ + list_for_each_entry_safe(b_desc, temp, &se_shared_mem_mgmt->mem_pool_buf_list, link) { + if (reclaim) + gen_pool_free(priv->mem_pool, + (unsigned long)b_desc->shared_buf_ptr, + b_desc->size); + list_del(&b_desc->link); + kfree(b_desc); + } +} + +static void se_fw_busy_work(struct work_struct *work) +{ + struct fw_busy_info *fbusy_info = + container_of(work, struct fw_busy_info, fw_busy_work); + struct se_if_priv *priv = + container_of(fbusy_info, struct se_if_priv, fw_busy_info); + + se_clear_fw_busy(priv); +} + static int se_suspend(struct device *dev) { struct se_if_priv *priv = dev_get_drvdata(dev); diff --git a/drivers/firmware/imx/se_ctrl.h b/drivers/firmware/imx/se_ctrl.h index dd4a1ea7e35a..ac5706b99a51 100644 --- a/drivers/firmware/imx/se_ctrl.h +++ b/drivers/firmware/imx/se_ctrl.h @@ -10,20 +10,42 @@ #include #include #include +#include #define MAX_FW_LOAD_RETRIES 50 #define SE_MSG_WORD_SZ 0x4 #define RES_STATUS(x) FIELD_GET(0x000000ff, x) +#define MAX_DATA_SIZE_PER_USER (128 * 1024) #define MAX_NVM_MSG_LEN (256) +/* Largest firmware response buffer size in bytes; equals ELE_DEBUG_DUMP_RSP_SZ (0x5c). */ +#define MAX_ALLOWED_RX_MSG_SZ 0x5c #define MESSAGING_VERSION_6 0x6 #define MESSAGING_VERSION_7 0x7 +struct se_if_open_gate { + struct miscdevice miscdev; + struct se_if_priv *priv; + /* to lock to update the structure */ + struct mutex lock; + struct kref refcount; + bool dying; + /* set once misc_register() has succeeded (deferred to probe end) */ + bool registered; +}; + struct se_clbk_handle { struct se_if_device_ctx *dev_ctx; struct completion done; bool signal_rcvd; + /* + * Set under clbk_rx_lock once a real response is copied into rx_msg, + * cleared when a new transaction is armed. Lets ele_msg_rcv() tell a + * genuine response from a teardown-forced complete_all() with no data. + */ + bool rx_delivered; u32 rx_msg_sz; + /* * Assignment of the rx_msg buffer to held till the * received content as part callback function, is copied. @@ -45,10 +67,46 @@ struct se_imem_buf { u32 state; }; +struct se_buf_desc { + u8 *shared_buf_ptr; + void __user *usr_buf_ptr; + u32 size; + struct list_head link; +}; + +struct se_shared_mem { + dma_addr_t dma_addr; + u32 size; + u32 pos; + u8 *ptr; +}; + +struct se_shared_mem_mgmt_info { + struct list_head mem_pool_buf_list; + struct list_head pending_in; + struct list_head pending_out; + + struct se_shared_mem non_secure_mem; +}; + /* Private struct for each char device instance. */ struct se_if_device_ctx { struct se_if_priv *priv; + struct miscdevice *miscdev; const char *devname; + u32 sess_hdl; + u32 strg_hdl; + bool cleanup_done; + unsigned long rcv_msg_timeout_jiffies; + + /* process one file operation at a time. */ + struct mutex fops_lock; + + struct se_shared_mem_mgmt_info se_shared_mem_mgmt; + struct list_head link; + + /* Add reference counting */ + struct kref refcount; }; /* Header of the messages exchange with the EdgeLock Enclave */ @@ -90,6 +148,89 @@ struct se_fw_load_info { struct mutex load_fw_lock; }; +struct cmd_rcvr_data_info { + /* + * Tracks the last FW export command received by the command receiver + * (ELE_STORAGE_MASTER_EXPORT_REQ or ELE_STORAGE_CHUNK_EXPORT_REQ). + * Set by cmd_receiver_specific_ops() when the FW command arrives via + * read(), cleared at entry. se_cmd_receiver_allowed_rsp() checks it to + * ensure write() can only follow a matching read() for export responses. + * Stored per SE interface (not file-scope static) to prevent a race when + * multiple SE interfaces (e.g. ELE and V2X) run concurrent export flows. + */ + u8 cmd_rcvr_last_rcvd_cmd_id; + + /* + * Export buffer size communicated by the FW in the preceding + * ELE_STORAGE_MASTER_EXPORT_REQ or ELE_STORAGE_CHUNK_EXPORT_REQ + * command. cmd_receiver_specific_ops() stores it here; se_val_cmd_addrs() + * reads it when size_idx == SE_CMD_RCVR_ADDR_VAR_SIZE to range-check + * the response buffer. Stored per SE interface so concurrent ELE and V2X + * export flows cannot corrupt each other's size. + */ + u32 cmd_rcvr_var_size; +}; + +struct fw_busy_info { + /* + * fw_busy acts as a circuit breaker: set when a synchronous + * transaction times out, cleared when the late FW response + * arrives and se_clear_fw_busy() has reclaimed the parked + * dev_ctx. + */ + atomic_t fw_busy; + + /* + * Serialise fw_busy, fw_busy_dev_ctx state updates between the + * timeout path, late-response callback/work, and teardown. + */ + spinlock_t fw_busy_lock; + + /* + * dev_ctx whose synchronous transaction timed out; parked here + * so a late FW response can still be routed to it by + * se_clear_fw_busy(). + */ + struct se_if_device_ctx *fw_busy_dev_ctx; + /* work item scheduled by se_if_rx_callback() on late response. */ + struct work_struct fw_busy_work; + /* + * Snapshot of the orphaned firmware response that triggered + * fw_busy_work. Written once (under clbk_rx_lock, before + * schedule_work()) and read once (in se_clear_fw_busy(), under + * clbk_rx_lock). Sized to MAX_ALLOWED_RX_MSG_SZ (= ELE_DEBUG_DUMP_RSP_SZ, + * the largest firmware response) to accommodate any FW reply. + */ + u8 orphan_fw_rx_msg[MAX_ALLOWED_RX_MSG_SZ]; +}; + +struct msg_excl_flow_info { + /* + * Optional exclusive reservation of the SE messaging interface for a + * single flow. A flow that needs ele_msg_send_rcv() reserved to itself + * (e.g. the fw_busy recovery in se_clear_fw_busy()) publishes its own + * task here via se_reserve_msg_if() from OUTSIDE ele_msg_send_rcv(), and + * releases it with se_release_msg_if() when done. While non-NULL, + * ele_msg_send_rcv() lets only this owning task through and rejects every + * other caller with -EBUSY; while NULL the interface is open to general + * se_if_cmd_lock-based message exchange. Written under msg_excl_lock, + * read locklessly in ele_msg_send_rcv() via READ_ONCE and only ever + * compared against current, so a stale read is harmless (a non-owner can + * never match). + */ + struct task_struct *msg_excl_owner; + /* Serialise updates to msg_excl_owner. */ + spinlock_t msg_excl_lock; + /* + * Serialises competing reservation flows. se_reserve_msg_if() holds + * this mutex for the whole duration of a reservation, so a second flow + * that wants exclusive ownership of the messaging interface sleeps here + * until the current owner calls se_release_msg_if(). Held only from + * process/workqueue context. + */ + struct mutex msg_excl_flow_lock; +}; + struct se_if_priv { struct device *dev; @@ -99,12 +240,14 @@ struct se_if_priv { * under se_if_cmd_lock. */ struct se_clbk_handle waiting_rsp_clbk_hdl; + /* * prevent new command to be sent on the se interface while previous * command is still processing. (response is awaited) */ struct mutex se_if_cmd_lock; + struct msg_excl_flow_info msg_excl_flow; struct mbox_client se_mb_cl; struct mbox_chan *tx_chan, *rx_chan; @@ -112,10 +255,40 @@ struct se_if_priv { const struct se_if_defines *if_defs; struct se_fw_load_info load_fw; - atomic_t fw_busy; + /* + * Set once teardown begins. New synchronous transactions are rejected + * and a teardown-forced completion is not mistaken for a real firmware + * response. + */ + atomic_t going_away; + struct fw_busy_info fw_busy_info; struct se_if_device_ctx *priv_dev_ctx; + struct list_head dev_ctx_list; + + /* prevent modifying priv member variable in parallel. */ + struct mutex modify_lock; + u32 active_devctx_count; + u32 dev_ctx_mono_count; + + /* Add reference counting */ + struct kref refcount; + + /* stable gate used by .open() */ + struct se_if_open_gate *open_gate; + struct cmd_rcvr_data_info crcvr_info; }; char *get_se_if_name(u8 se_if_id); +void unset_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx); +int set_dev_ctx_as_command_receiver(struct se_if_device_ctx *dev_ctx); +bool se_is_fw_busy_ctx(struct se_if_device_ctx *dev_ctx); +void se_dev_ctx_shared_mem_cleanup(struct se_if_device_ctx *dev_ctx); +int get_shared_mem_slot(struct se_if_device_ctx *dev_ctx, + u32 *length, dma_addr_t *ele_dma_addr, void **ptr); +int se_get_mem_pool_buf(struct se_if_device_ctx *dev_ctx, void **buf, + dma_addr_t *daddr, u32 len); +void se_cleanup_mem_pool_buf(struct se_if_device_ctx *dev_ctx, bool reclaim); +int se_reserve_msg_if(struct se_if_priv *priv); +void se_release_msg_if(struct se_if_priv *priv); #endif diff --git a/include/uapi/linux/se_ioctl.h b/include/uapi/linux/se_ioctl.h new file mode 100644 index 000000000000..c7ce736b5ea4 --- /dev/null +++ b/include/uapi/linux/se_ioctl.h @@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: (GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause*/ +/* + * Copyright 2025 NXP + */ + +#ifndef SE_IOCTL_H +#define SE_IOCTL_H + +#include + +#define SE_TYPE_STR_DBG "dbg" +#define SE_TYPE_STR_HSM "hsm" +#define SE_TYPE_ID_UNKWN 0x0 +#define SE_TYPE_ID_DBG 0x1 +#define SE_TYPE_ID_HSM 0x2 +/* IOCTL definitions. */ + +struct se_ioctl_setup_iobuf { + __u64 user_buf; + __u32 length; + __u32 flags; + __u64 ele_addr; +}; + +struct se_ioctl_shared_mem_cfg { + __u32 base_offset; + __u32 size; +}; + +struct se_ioctl_get_if_info { + __u8 se_if_id; + __u8 interrupt_idx; + __u8 tz; + __u8 did; + __u8 cmd_tag; + __u8 rsp_tag; + __u8 success_tag; + __u8 base_api_ver; + __u8 fw_api_ver; +}; + +struct se_ioctl_cmd_snd_rcv_rsp_info { + __u64 tx_buf; + __u64 rx_buf; + __u32 tx_buf_sz; + __u32 rx_buf_sz; +}; + +struct se_ioctl_get_soc_info { + __u16 soc_id; + __u16 soc_rev; +}; + +/* IO Buffer Flags */ +#define SE_IO_BUF_FLAGS_IS_OUTPUT (0x00u) +#define SE_IO_BUF_FLAGS_IS_INPUT (0x01u) +#define SE_IO_BUF_FLAGS_USE_SEC_MEM (0x02u) +#define SE_IO_BUF_FLAGS_USE_SHORT_ADDR (0x04u) +#define SE_IO_BUF_FLAGS_IS_IN_OUT (0x10u) + +/* IOCTLS */ +#define SE_IOCTL 0x0A /* like MISC_MAJOR. */ + +/* + * ioctl to designated the current fd as logical-receiver. + * This ioctl is send when the nvm-daemon, a slave to the + * firmware is started by the user. + */ +#define SE_IOCTL_CHECK_CMD_RCV_REG_STATUS _IO(SE_IOCTL, 0x01) + +/* + * ioctl to get the buffer allocated from the memory, which is shared + * between kernel and FW. + * Post allocation, the kernel tagged the allocated memory with: + * Output + * Input + * Input-Output + * Short address + * Secure-memory + */ +#define SE_IOCTL_SETUP_IOBUF _IOWR(SE_IOCTL, 0x03, struct se_ioctl_setup_iobuf) + +/* + * ioctl to get the mu information, that is used to exchange message + * with FW, from user-space. + */ +#define SE_IOCTL_GET_MU_INFO _IOR(SE_IOCTL, 0x04, struct se_ioctl_get_if_info) +/* + * ioctl to get SoC Info from user-space. + */ +#define SE_IOCTL_GET_SOC_INFO _IOR(SE_IOCTL, 0x06, struct se_ioctl_get_soc_info) + +/* + * ioctl to send command and receive response from user-space. + */ +#define SE_IOCTL_CMD_SEND_RCV_RSP _IOWR(SE_IOCTL, 0x07, struct se_ioctl_cmd_snd_rcv_rsp_info) +#endif -- 2.43.0