From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 688DCCD6E60 for ; Mon, 1 Jun 2026 10:49:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ku/LApgaR/nfFXgkmtmdfuvWgOaL8+Tqey9W6gHGJNM=; b=xE6FEKitHWTPzjAPcdp1F3WtF7 8KztwYBvGDxLMqdHKF0KTpJGN5gYpqoI3lHUf8s99TWe6fHycvIYdlaukqQnQYRveBCFy6rSD7cQ+ WB8yIFAGSqt1R3A04lz7Fm7spUBUcLMeOdwwbD4w2/8bubYS1vZIyL5MEpA2EE5NnmEKvjNVp4L1V sIrAgdKzKtAvcjOCSH08HAsze1VEW3r0OdKbCfPwT+24UxS+CGovQ13DTvXRVX5izr6oCgM2oNHQK fhDx4y7yuPxFX3fIShOr4a+JhGzt3dOiLoGFyDtzNBY5Z+ocQMZu1YwnPMY3ko11vT72ljFSgR79W u2kKyTJw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wU0Co-0000000Aesl-2oEs; Mon, 01 Jun 2026 10:49:14 +0000 Received: from mail-westusazon11012000.outbound.protection.outlook.com ([52.101.43.0] helo=SJ2PR03CU001.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wU0Cm-0000000AeqU-3srI for linux-arm-kernel@lists.infradead.org; Mon, 01 Jun 2026 10:49:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LdIU0NemxS7pxistCZHkMEDgvQLHmryFFBOn6b/LT9N72GdjA029PlLOfhFzKytvGvBmPDjtvZ+WbKjUeyI5TZmi144CU+VPYO1DJb5/RbIvet6R8rXzvO68FF9WmOp1HVLe2qai/T0M9A/vlFeS/F+r4grrnwBy2upmPsLEIM2JjLQDRZV9aHs8GOy239M61Z7ey4tgSDnfBUCTgqY4MpXVlUA5RosKOy37pQuiXi4i+AiB612YYOPiTFRewkn2fQv97fipWtQDoFHGLgsf5CbSWEeNs6k8u2wwnqxscuxMhqXkHPjFI3xViejEcRc1isEXS32IomYQ4R1tkqhgZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ku/LApgaR/nfFXgkmtmdfuvWgOaL8+Tqey9W6gHGJNM=; b=iJzrH/poGUL46mMVn41FbTLocx+0iOXet2TxnHqGWG5xJwdPeFdhuaJoNZH1hFIIDiva0PPoqe2evIkNSm3nVZcJ6gtE2l5CdL66KX71UZxLlfiyPANGT/UMn4u2BIPC5+9ZsQ59og8gGuqb5qFfz+iOuhG72J78j4NYzTBO0E4GWDvSNLjkYJZCNxFifJbvGQmLgHVksn67Mh0KUjpgl9jS0FIwiPw7pFgJllmmEl9OtM2W/nsJ4dz66o2YriBegmdsdUHA9OKAWkONIjyIKEKmqiKAnGH1SI8byZBhvfuEReOxHsh03y6z29hfR+FPebu87ZTlUYB+EbYshgp8uQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ku/LApgaR/nfFXgkmtmdfuvWgOaL8+Tqey9W6gHGJNM=; b=gWkCJLivHxRrUyG4oLe/yFXIFdqyogz5jO1L7VYM0b47By7D+kcrH2ZO4Kzuf5HhCtAcYVpB0+wRl4b/X/Cwn0WB4MJqjaOvX7ehcKN28lbnsCJRDX56PvvWkF84tRjRgoEopdYo5fgyFKm4zHyj7oBCiB6V0cMrUTeyZqAgEVHXoNCf0l3Cqrr7DyQ9ZstjOpGOtiJZQnEwABY61R+GFmZYmzId7C21yEc3e9qYprfLXki48JMSPGByRuCa9UShGhdUdcCRH/ZU2Wa0wrgq7gt3fKGJVnlRtvF69eS8rOXryzidUKCWKJYo59uVCvLenTD1RS05opavrA6aieKm0g== Received: from SJ0PR03CA0229.namprd03.prod.outlook.com (2603:10b6:a03:39f::24) by CH2PR12MB4296.namprd12.prod.outlook.com (2603:10b6:610:af::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.16; Mon, 1 Jun 2026 10:49:05 +0000 Received: from SJ1PEPF000023D4.namprd21.prod.outlook.com (2603:10b6:a03:39f:cafe::11) by SJ0PR03CA0229.outlook.office365.com (2603:10b6:a03:39f::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.71.16 via Frontend Transport; Mon, 1 Jun 2026 10:49:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SJ1PEPF000023D4.mail.protection.outlook.com (10.167.244.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.0 via Frontend Transport; Mon, 1 Jun 2026 10:49:05 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Mon, 1 Jun 2026 03:48:51 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Mon, 1 Jun 2026 03:48:51 -0700 Received: from build-amhetre-focal-20250829.internal (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.2562.20 via Frontend Transport; Mon, 1 Jun 2026 03:48:51 -0700 From: Ashish Mhetre To: , , , , CC: , , , , Ashish Mhetre Subject: [PATCH v3 3/3] iommu/arm-smmu-v3: Issue CFGI/TLBI twice on Tegra264 Date: Mon, 1 Jun 2026 10:48:45 +0000 Message-ID: <20260601104845.995005-4-amhetre@nvidia.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260601104845.995005-1-amhetre@nvidia.com> References: <20260601104845.995005-1-amhetre@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public Content-Transfer-Encoding: 8bit Content-Type: text/plain X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000023D4:EE_|CH2PR12MB4296:EE_ X-MS-Office365-Filtering-Correlation-Id: a9ba1784-c6ad-48bd-97c4-08debfcb6670 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700016|376014|18002099003|22082099003|56012099006|6133799003|3023799007|11063799006; X-Microsoft-Antispam-Message-Info: /t/gmrucF4rUxfPTqI4PqrcQmiTBNKVm+LEy8JJfz6RTlwOALkquJ/VCUQb3JGo6qZyidPc8nb4Hw+WntQXNr6IYuKqaEuu5Sohv0eO/u2kpLHCGg0xQCGU6v/8Eyib2pMUAJju6B0SxrORQEQf72BcbhYHM8m8W8PMw8GzkoMKB8PW4KcHlA1i/4F+Q3c/rc6E4010Ow+TMTeKltUkKA+mP3efPDhsvN9HLT8Sh4yN8MpSCNTRj2y5pTCr/2M76sS8iOLDtnGeRcOLiksv3K9WgVcl6P7qkauC9sdJ5uhdyxyTP1mNc23kJep9iHzxccrgseWzlNRVfmmgD4HBa3wJYa+iVPAwhro58bQnHDFJYeC3PhE2GDqf/t3NvvTrGwCUylJ6Z8AYQvUXE+oxi39t757bok3lo3d92Hx++lWBRtVccXy44ZAMcKLT2gU+j/0QeknGwS0Ne1LS66YuTVQasgKd2hHMsLTWY10+snJFEczJd4XEvioW3XzKzgEjbVH3S2PaGH/e4Y2QPZRp+JnH/TsE70Ozd0SUANv9ToRtuBJvopGsM3x5NKG0Equcih0NaAHSFQNuXda8ft4gEGmXB/8PXAR2UPIw3p+acwehCMYYWZn6hJR1E8zGi1woXD5QMtEkuJwYEjhg6wxQpq+5mzY8GEiWORyaARM9IJmwShkHlYnq57AMSi3LHc+H+oaaGqOj2kTJx97cAF3eKdrWXnVGJ/KuTlevjes0Ft8E= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700016)(376014)(18002099003)(22082099003)(56012099006)(6133799003)(3023799007)(11063799006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QvI5uEeW7ZA9Y+kjsTvbAS46FIaKT/k2XwS12TEVvY/6HqDYK/30fN06hVCR6EHPJCupMNklDMFAcDaJjOqzV/b6PV03oofOnlUf8+9Lpn4a8WGHboHCAF8n0+Ivldj1CFk1pbkSRTbmm+Jz4Ob0+AF9xeepIA+0bQTlNWebEwUCfBKF/qSQunIWPv/ZWokMmSfvzddDXg7wu+bWFg4yWROL4fbZYdCFPMGw8nY1KXmbFpAo0oHDtwho9YHgocgwqO0BujzmJh6+0L+hFsi9/ynUEsndZ1KQ4BFdeV6dg4rTlu39O6zpYh83SclyrbIzeF1TLfE04NWeCkTJ0/gQJlXB8szjZ0hhzESZH2hPcCuhpaUouz+6hi1AA3kKNPmiv+Qcov6Csh93JuEDrZjrEjU/aUXLVlGa/0bA9HPXzp7JmFTUhfvH8E0p5WSohqmB X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2026 10:49:05.0776 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a9ba1784-c6ad-48bd-97c4-08debfcb6670 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000023D4.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4296 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260601_034913_001619_B145C6E8 X-CRM114-Status: GOOD ( 16.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apply the workaround for Tegra264 erratum ARM_SMMU_OPT_TLBI_TWICE by issuing every CFGI/TLBI cmdlist twice on affected SMMU instances, with CMD_SYNC after each. The erratum requires this exact sequencing: TLBI/CFGI ... CMD_SYNC TLBI/CFGI ... CMD_SYNC Rename the existing arm_smmu_cmdq_issue_cmdlist() to __arm_smmu_cmdq_issue_cmdlist() and add a thin wrapper that, on affected SMMUs and when @sync is true, re-issues the same cmdlist a second time when arm_smmu_cmd_needs_tlbi_twice() is true. For the in-tree batching path, register the Tegra264 condition with arm_smmu_cmdq_batch_force_sync() so that a full batch carrying CFGI/TLBI commands flushes with sync=true. For iommufd VSMMU path add an arm_vsmmu_can_batch_cmd() predicate that splits the iommufd batch at cmd which doesn't need doubling. Suggested-by: Nicolin Chen Signed-off-by: Ashish Mhetre --- .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 23 +++++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 36 ++++++++++++++++--- 2 files changed, 54 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c index 1e9f7d2de344..78c96a2b652b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c @@ -350,6 +350,26 @@ static int arm_vsmmu_convert_user_cmd(struct arm_vsmmu *vsmmu, return 0; } +/* + * On Tegra264, arm_smmu_cmdq_issue_cmdlist() doubles every CFGI/TLBI + * submission (see ARM_SMMU_OPT_TLBI_TWICE). The doubling decision is + * taken once per cmdlist based on the first command, so a single + * batch must not mix commands that need doubling with commands that + * do not. Split the iommufd batch whenever the next user command + * crosses that boundary. + */ +static bool arm_vsmmu_can_batch_cmd(struct arm_smmu_device *smmu, + struct arm_vsmmu_invalidation_cmd *last, + struct arm_vsmmu_invalidation_cmd *next) +{ + struct arm_smmu_cmd next_cmd = { + .data[0] = le64_to_cpu(next->ucmd.cmd[0]), + }; + + return arm_smmu_cmd_needs_tlbi_twice(smmu, &last->cmd) == + arm_smmu_cmd_needs_tlbi_twice(smmu, &next_cmd); +} + int arm_vsmmu_cache_invalidate(struct iommufd_viommu *viommu, struct iommu_user_data_array *array) { @@ -382,7 +402,8 @@ int arm_vsmmu_cache_invalidate(struct iommufd_viommu *viommu, /* FIXME work in blocks of CMDQ_BATCH_ENTRIES and copy each block? */ cur++; - if (cur != end && (cur - last) != CMDQ_BATCH_ENTRIES - 1) + if (cur != end && (cur - last) != CMDQ_BATCH_ENTRIES - 1 && + arm_vsmmu_can_batch_cmd(smmu, last, cur)) continue; /* FIXME always uses the main cmdq rather than trying to group by type */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 08684bd40a6d..f38c21b56f28 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -698,10 +698,10 @@ static void arm_smmu_cmdq_write_entries(struct arm_smmu_cmdq *cmdq, * insert their own list of commands then all of the commands from one * CPU will appear before any of the commands from the other CPU. */ -int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, - struct arm_smmu_cmdq *cmdq, - struct arm_smmu_cmd *cmds, int n, - bool sync) +static int __arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq *cmdq, + struct arm_smmu_cmd *cmds, int n, + bool sync) { struct arm_smmu_cmd cmd_sync; u32 prod; @@ -820,6 +820,26 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, return ret; } +int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq *cmdq, + struct arm_smmu_cmd *cmds, int n, + bool sync) +{ + int ret = __arm_smmu_cmdq_issue_cmdlist(smmu, cmdq, cmds, n, sync); + + /* + * On Tegra264 (see ARM_SMMU_OPT_TLBI_TWICE) re-issue the same + * cmdlist with another CMD_SYNC to satisfy the erratum. + * Callers must ensure the batch carries a uniform opcode class + * so that checking the first command is enough; the iommufd + * VSMMU path enforces this with arm_vsmmu_can_batch_cmd(). + */ + if (!ret && sync && arm_smmu_cmd_needs_tlbi_twice(smmu, &cmds[0])) + ret = __arm_smmu_cmdq_issue_cmdlist(smmu, cmdq, cmds, n, sync); + + return ret; +} + static int arm_smmu_cmdq_issue_cmd_p(struct arm_smmu_device *smmu, struct arm_smmu_cmd *cmd, bool sync) { @@ -863,6 +883,14 @@ static bool arm_smmu_cmdq_batch_force_sync(struct arm_smmu_device *smmu, (smmu->options & ARM_SMMU_OPT_CMDQ_FORCE_SYNC)) return true; + /* + * Tegra264 erratum (see ARM_SMMU_OPT_TLBI_TWICE). The batch holds + * a uniform opcode class, so checking the first command is enough. + */ + if (cmds->num == CMDQ_BATCH_ENTRIES && + arm_smmu_cmd_needs_tlbi_twice(smmu, &cmds->cmds[0])) + return true; + return false; } -- 2.50.1