From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 329ECFCC074 for ; Fri, 6 Mar 2026 20:19:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:CC:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5wQRgVzOHKbSno0+gTSSZ1eLOZzfc5dxI+Csjpl5omQ=; b=p12HL3J6fNXz2X9D/5UeFEadVo 9nLPd0M5M7g+a3Bst6dWnyCLz70yq7DBtv1hWr+h2nATavIXa6NkfpzGVOFs3Oh0WANS/MC5urZWg 1HsCy/SZL7ERJcAaiIbweOC8Dpt8ajji1Lzw6Cr//TJ8nonv66A+E5rRhGA7AjGJ97GjoZMO95ews GEsc9ZOj4VkrKh92jgmalNTct43zLZ/MKWQEcmg0YOHQ0K66KVpFBUuTYKmZ6jTzSQm86Yny69RHm W3wKHkPyw9WmyeIyoYWlRqe+4/rzZTu7L4A+yUasYJo2oID3c/IBJw6q0u5tthQLQvfqvr1hSLvRQ oqHBEz8Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vybdU-00000004SwM-27RH; Fri, 06 Mar 2026 20:19:00 +0000 Received: from mail-westus2azon11012017.outbound.protection.outlook.com ([52.101.48.17] helo=MW6PR02CU001.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vybdS-00000004Sw1-1Zy0 for linux-arm-kernel@lists.infradead.org; Fri, 06 Mar 2026 20:18:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wIDJtyeFXlMbtX30BxMLI+733IP6U8nv6ipCRDMRvFXbwlPZVLLuyg0hH2J/sstmJ3n0I3FuPM+C+LZX6sjye1PaRWj0VlZTeee5DsXrk3WVRYzLNzRcsGRceZcWVtOlJg25qBqsGz5nyaWzdttxogBjzM3JtK9/nVgwfyQt8mkvSZo3AB/KsAQJpMSjRzOhjr0VKvVT9MaOJwyiET3LR5CshuGYIJjDl3J9rDqnw1G54F9omurv7htr1BxY9PcTuZSV9FWs2/CHmOrlXWeUXgZgP+MQl8FO/gSiPK+AnnKmnYQu0cmRKQRG6mjtj76HbsaZK+9OjF1kBsGh8jMsyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5wQRgVzOHKbSno0+gTSSZ1eLOZzfc5dxI+Csjpl5omQ=; b=ABeTufe38vF1G+laoujnr+PYUEH9uVF8iYoMF5tJd9x+uUoIw5meqFnr5sZuRDOMYxvULQXfK9dJJCcQw79PcvOHE4cMhzDymQkFE+DJXIw7W6zEOsmRwY2F9Aee0wk5hbvN+8Akty6+NRr3/iYYSHKF48C74Jpv/MMIvgUxEM189ztwd5RnV04o+2iJwQbMWPR05Vs6df1jMWT5ojAzSIEocUFhRelzejfm97ufKM6JYHh6RZV9uWRAKCDFqcVUvDliqoDrjtkroEVAKqqMPQ+imISpxBpi7M775duIOvLyfGGNej/gRbG1DA5xlaNI3Tq+G83e3wgZnQHQnlwMSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5wQRgVzOHKbSno0+gTSSZ1eLOZzfc5dxI+Csjpl5omQ=; b=L6QEx1Q4WMETqK4J+WPkbXDdDFu6bv4C2RbkY5F7pINoKVGHi+Mp7lPpi2UeU+rrydhZTblRw3wmSx6n7wV2OvSLnFR79LmglepWnXxUXGfhEiJ7BRLW7G0dkUknVeoxuJLMeG6IVZoivqp0b55d8+AulExcesgCisAWJYo/zyijIYUyudk1dAPhsa9kDeHCfUFu0w2HxwEES7l7iy9xy4GR9NyZ5wya+w/jd6FRdVQABHC+wYdbbavZ299crZ4S4ikd5Hr0a6e0eeIggzs10y23fxBezVpw871I9lkx0VwlaNdbBmxVJaTIry24uUU9xikjpT+fGHp0NAv0KaXIOQ== Received: from MN2PR14CA0026.namprd14.prod.outlook.com (2603:10b6:208:23e::31) by LVUPR12MB999160.namprd12.prod.outlook.com (2603:10b6:408:3a4::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9700.5; Fri, 6 Mar 2026 20:18:48 +0000 Received: from BL02EPF00021F6E.namprd02.prod.outlook.com (2603:10b6:208:23e:cafe::dc) by MN2PR14CA0026.outlook.office365.com (2603:10b6:208:23e::31) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Fri, 6 Mar 2026 20:18:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BL02EPF00021F6E.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Fri, 6 Mar 2026 20:18:47 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Fri, 6 Mar 2026 12:18:26 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Fri, 6 Mar 2026 12:18:26 -0800 Received: from Asurada-Nvidia (10.127.8.10) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Fri, 6 Mar 2026 12:18:25 -0800 Date: Fri, 6 Mar 2026 12:18:24 -0800 From: Nicolin Chen To: Jason Gunthorpe CC: Robin Murphy , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v1 2/2] iommu/arm-smmu-v3: Recover ATC invalidate timeouts Message-ID: References: <20260305153911.GT972761@nvidia.com> <20260305234158.GB1651202@nvidia.com> <60d77adc-d5a6-40e2-a497-a57004f23e7e@arm.com> <20260306140115.GH1651202@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20260306140115.GH1651202@nvidia.com> X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6E:EE_|LVUPR12MB999160:EE_ X-MS-Office365-Filtering-Correlation-Id: 92d62d8b-bf8c-4e84-14b2-08de7bbd9315 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|82310400026|36860700016; X-Microsoft-Antispam-Message-Info: 9ZBg86uxa+qzYxOFgFi/iX02Z2CB/snhBlTAk5CZkojXZgYzRhGTgIAwcQ44xYZKQpzoIyBoV4XOCbGau2iL2O8/beT5pSYvKw2hkOPMz2ADnb7Y/O+nOhwTWg5MR7R5vFZLbnEMNccAZFfLvqDrsuYU1Rc7VeOLW+7Jyz808mxlTWNjsuXF7jCGudHpa/vbhk0xgsODHKNU064YSYNXQYpYYA8811KadKOy103p4UvkktBg5bPW0QrxJ2z/GRjPbns6fZLf9kZXTXWIxCgfR8SMOfPyBQFY4i2+EvcRS4ApkIwvA2U10b6ln9AqIX3w+YpqWfbqw6e0Oeqd3tSDP28Zzhey8hZ3dXFjWCVEvOZKmT9I3LeO4kpdO8k6Guff2G3NpFiQP6bBOBg2nKko8v/4J/W+xAbq3Mt6uIlCBflb+5Cz1kjGskCFgnhDGb10rl9V6j+t6eEKqBIhag53aFIXi/lJj/taHEdvIaFwht2laAFRCMLr5SGP+ZqZVXEUrNrWkxAu+o/7ybc7lENorGJ371W3qflHgmhWCnl8kotRHHeBDp8Ye3oTsDxq3mvO9xNh+pwmKzgBj6AMmX5YDT3r3rDsN79utecczgLdg+Gy6865cVyj4t6oRgr8FTFH5xVQYywy2lnoVERkunZfGsLOcY6pJGnQIW6F6/QB/EiTs+P+S1cYo2/0hjq60PWex4bdUtf4Q1BYj7i0fu6L4j4eP3IQ4mcDNJr6tMTqAp7werX5CUFmK54pqR2ObsVf65jiXH3t9KVJYGcAoO6NZw== X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(82310400026)(36860700016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uWEm9Fw9p3E8tGgi1yaPJHl4SnAd02EwdNKbbdTrjXtQ5QGUJh+2qjIbCXI6sONF69q5epdZnhXuq2hQ8KohDXGYzh6cVpWratTlAyuEdY+noYXQbtB0Sb5wr+hmznPVdE+pAOlFQ2Th5w8tEBjdadBrrsp4vEqxhn6u4eah/ZLS1LwPYE9doqYV7uQuBxNo/HUrny99MtFuw+OJQMb7a4jBgZ3QulK06hErTUCZLHUNHIXzoXNIh51HsbHy4ZmQQPW0DabwHdIoP6TXJon+iZ+Nhxqbsqg77ZMsWNd6dtWSVNTMmrhTAsNjgwk+iYni4N13qfi20i6L3hdmlar32NTpwdCXIY0QVyX+bhpnMSh8TV5MnJkSkgHiEpdjegkAdigqhcy1aeSvljbIcR4MA8rFE2FP8WKq8qEiX12rCeoMlHsuZEUas4vIqN535zgK X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2026 20:18:47.7864 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92d62d8b-bf8c-4e84-14b2-08de7bbd9315 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LVUPR12MB999160 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260306_121858_440365_F21397E3 X-CRM114-Status: GOOD ( 25.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Mar 06, 2026 at 10:01:15AM -0400, Jason Gunthorpe wrote: > On Fri, Mar 06, 2026 at 01:22:11PM +0000, Robin Murphy wrote: > > On 2026-03-05 11:41 pm, Jason Gunthorpe wrote: > > > On Thu, Mar 05, 2026 at 01:15:45PM -0800, Nicolin Chen wrote: > > > > > > > You mean in arm_smmu_cmdq_issue_cmdlist() that issued the timed > > > > out ATC command? > > > > > > Yes, it was my off hand thought. > > > > > > > So my test case was to trigger a device fault followed by an ATC > > > > command. But, I found that the ATC command submission returned 0 > > > > while only the ISR received: > > > > CMDQ error (cons 0x03000003): ATC invalidate timeout > > > > arm_smmu_debugfs_atc_write: ATC_INV ret=0 > > > > > > > > It seems difficult to insert a CMDQ_OP_CFGI_STE in the submission > > > > thread? > > > > > > I didn't look, but I thought the CMDQ stops on the ATC invalidation, > > > flags the error and the ISR NOP's the failing CMDQ entry and restarts > > > it to resume the thread? Is that something else? > > > > > > If so you could insert the STE flush instead of a NOP > > > > Nope, sadly the timeout is asynchronous, and CERROR_ATC_INV_SYNC is only > > reported on the *next* CMD_SYNC - it can't even tell us which CMD_ATC_INV(s) > > had a problem. > > !! That's a good point! The new invalidation code runs many ATC > invalidations under one sync to optimize for SVA performance so we > have no idea what devices need to be reset :( > > So we really do need to signal to the issuing thread and it will have > to go back and check how many ATC invalidations are under this sync > and re-issue one by one to isolate the error then issue the STE change > and sync. Nothing from an ISR then.. IIUIC, we would have two timeouts to identify the device(s), so we wouldn't need to give away the optimization of batching ATCI cmds? Will letting a faulty device time out once again give it a window to corrupt the memory? Thanks Nicolin