From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EA691C5CFEB for ; Thu, 13 Aug 2026 14:13:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Mn6Vh1o932dhUDkIRW3So4IeXLZqoga7DH3eYMz50/s=; b=JAIQ/kx3KWx04Pg58picRZ+PDx 0BQH77pIIFWmku7yKHm5kvvL51QddQirhz8J50P0nibSjveWbg1yumHZWkY8sSUh3V9a9/3WcKimL LgyxUo44+kjxsK2Dv/SWXH88xx3+OFjFPFR0euNTEm6JGpxjXHwTH5IwD02zJVYz8k3wQD75Gx2zO 4CxEXccFztI3Pld5RLjHXEZsr+cyQDntZWSLa39H6zPr4GUvnlxRppF4SnNfuHYwFb8fM8VKcq9GG ztU3UuwsL1LaFqPWsrmHn2dDnAFOJRi33tBrE8G1aPCEDG0kHGslNnvYnQWybCUR5lretUsgYb+8k SqzbkZRg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wuWBY-00000000p4M-30Cf; Thu, 13 Aug 2026 14:13:32 +0000 Received: from mail-southcentralusazlp170130001.outbound.protection.outlook.com ([2a01:111:f403:c10c::1] helo=SA9PR02CU001.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wuWBV-00000000p3S-31gn for linux-arm-kernel@lists.infradead.org; Thu, 13 Aug 2026 14:13:31 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tXIEZm3yH8SBJnBCukpSqa6vTswGi3c8NZyB/EEx1gLJgQEXpcQf3dze9NgqGvSYkUmRX/wlKRbEOXMtQM5Aa++yrFox1GR8XRrRzBwelZ13y+8Xz3NamAlXJseKvUgtj5YOxTxuFkcQFakz5hqk3/vudGatGkb0xZUGYN+/zLcYCOt79WoMRCSvjyEygUrDV5H/bPy/NQyoN0Sk6X18J8mkXJNr7TEhvcd6uA9Wpf6B4kyTt1Vn+W76MQhVD9RYacwGSdecIZ2DcPjTulVwVtUw8WcwAWzBWmDO6MMWhlwVuLqW4o9nu+z938oi5Zjbmy1dTtwzE5lKDTvaby/0fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Mn6Vh1o932dhUDkIRW3So4IeXLZqoga7DH3eYMz50/s=; b=njWD/9QXR0aYcfSGwwGmtMJU1KlRXWWEUG9zNn8Fm2qFWRmrOHzCcmtumrEZ2QOCdKc1rN4eta7JbPXmaqHv0RwXB0KcddZ/xVNSioGbdNfv0JLfzJa50iRMtxfmSEsPtMuqYe1tYZWOjQKOrwOUyfCnJ5pn/lSH3ncJIjBA1DRcFY7RSAszuEDUsENaEoDtpt+9aM4iuOza8U25aRpP8qbWxACBtsPL4tAgj1R5hXuTpIR/zeIznmmeEIlxhr856Y8t7s0F1ezPFOz5dcy58iDBm9OmTJMmTsqQS9zk2AHQQRRzqnlqO40ZJo7XnwuM/JKnoe6IGue1YYzqzjdsjw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mn6Vh1o932dhUDkIRW3So4IeXLZqoga7DH3eYMz50/s=; b=BVpbCQZTjmWATnt9OONDRbcFTN+hcclvtOXCPfucZQNmYnFCdtkOzGz7w0nBJaXoOOWVPPE4+EE3/V8kaqcJjaHYGz0/nHRShreDwPnZpcSkX+cLqgaLHBO6fODFJi7keAkXpnZcpKxhE3IwsaQdEE5PmeAiAva31qPHESctwMqkErTdLrF5JnNIGHZ382J/J+yLRMcOSjsIdnG6r5l+mQkP0Nr5f83WiFpjkRNi66ZmTWFRmxc/QhRf+0CeO4RkSmpnmuvt74qPnaQV13wyHt+Aza1hXMo+/5ImiRlQtdsN0QQryg2XqNE5AUBmollktvp0ayzxoByvODR8oFBZOg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by SN7PR12MB8772.namprd12.prod.outlook.com (2603:10b6:806:341::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.25; Thu, 13 Aug 2026 14:13:21 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%4]) with mapi id 15.21.0315.014; Thu, 13 Aug 2026 14:13:21 +0000 Date: Thu, 13 Aug 2026 11:13:20 -0300 From: Jason Gunthorpe To: Mostafa Saleh Cc: iommu@lists.linux.dev, "Joerg Roedel (AMD)" , Jean-Philippe Brucker , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon , David Matlack , Pasha Tatashin , patches@lists.linux.dev, Pranjal Shrivastava , Samiullah Khawaja Subject: Re: [PATCH v2 3/8] iommu/arm-smmu-v3: Optimize range invalidation for latency Message-ID: <20260813141320.GF730363@nvidia.com> References: <0-v2-43074a57a53a+fb95-smmu_tlbi_jgg@nvidia.com> <3-v2-43074a57a53a+fb95-smmu_tlbi_jgg@nvidia.com> <20260708001058.GA422027@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL1PR13CA0316.namprd13.prod.outlook.com (2603:10b6:208:2c1::21) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|SN7PR12MB8772:EE_ X-MS-Office365-Filtering-Correlation-Id: 6bb8c9d6-30fb-40cd-ca87-08def94507cb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|23010399003|366016|1800799024|22082099003|11063799006|56012099006|18002099003|4143699003|10067099003; X-Microsoft-Antispam-Message-Info: giELf6W9YNSQs7oIewePi6IsVKgqodwt/3/nyugQ6xwx8OEAAZVi92FnaNeBXrednBQcmfwh8XXSIV/v1w/UvRt3IVAR3jH1MldNi+KSTLw6ZZrpMZgdIlFnRR5L1a1/2Tp48+Bzv/rQ510Naq3WBf4Yfip5b04TlbX2JAlT/GfckJ8AR4xEhJFQxmEHZ7JfbYRaj1iXTu/UtMl4VzWjmCxYT26Pavu4IdEAQS3VHOyt5UTbDQLaynQwUfM/AOaIyhbaPMBaxXLhL0SCvKmPXCAInnuhckyqarsN+dwSCmqgoUHbnniX1QugzBMi1t5lxjL53J6sVpeAwMjvja7K4IdYq0ekoDJhFEqS39pNh3wWso1ybVM7DLvZWbwF+CL15EXUnMSDOZ9oTGQNpiY43+hqSEfffV+1RXD9ruaOPT7YrQOnAjfQrRTY5h4NUdaUE2tA1YM7fNdggDpB9Oi0PZfWR1QoNPzrrOvTB2Y+nEPJhc7XMqXSYIFCwGVVxSXvOM5zslZtYNveIDZ19sz3ZMrLp5oNV8ALV57dKefkiwNKMkuVe7M7sfj7lgtLUehakjp4E5eqs5utlbk81zQLFYsbZEkUzsmd//0PJnBiPZ7/AQDjIJDujIriOa8FVPdcN/Wx/f8zQWzLa5NhI/G9giH4cl4nEXBC0t+Dvm5g3BI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(23010399003)(366016)(1800799024)(22082099003)(11063799006)(56012099006)(18002099003)(4143699003)(10067099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ms3WiUNO162t/0N55h0XDJcKBeDGnsYPfOMpup3jb5RGSSDulrORFfyXMivS?= =?us-ascii?Q?NDVeve72qm8BgNMMfoh0bHyu5QCqm82T0FE9RFnEpLUEXm2HO5Lg9GnB2PCv?= =?us-ascii?Q?NpT1BLr4XVFFjEu0kHgQVydZcPHTQvrnkEyiGJQC1rpuqUi/HaoqrmIFqlpu?= =?us-ascii?Q?2EqAbUdKV0qWUJUjYDSo8vLpV0AlPCuWHqUk6DMnlY6TBGS5h37/+fbqyLDy?= =?us-ascii?Q?XPA9DlJjrPb3EbxYmYjYA4RhLq8iv4XI7InLhZv4ieN2sVvtFItf15SpmEiL?= =?us-ascii?Q?QAdmTLYXubavh/y4nfAtvoe9MGH6U0VjC4kR4q6LvKFZFEIemOIf9QK3d+3E?= =?us-ascii?Q?AVXm5S/uY71NkirD23E1k0uCpQJGoHqrbGFt+HVc+DYq41ApSue5a6A2qiDe?= =?us-ascii?Q?dBgx2bvQqshlDZMp162Lttp22vJw117WxVPBVtl+qALc/M88jEB+4G3bD2IM?= =?us-ascii?Q?miVd9dvEXmFbiSPz+CxYRQT73rKviQ2xI41ov552I33VdUE3kP6VnD3tsDZv?= =?us-ascii?Q?3TrHuxWzUmEjTvD9TNu4zeTGahkEluQaY0T0iLaBeIhagjAIdOjJCdT57HHO?= =?us-ascii?Q?KUrcbVik5pXPIM2r8LYNwWukinpTF8YZZYgQQDznK7BCTEHDxhjafAsFtfBh?= =?us-ascii?Q?DvUUnUC6FpmBSRlA45g9EO7cm4+Y6fTEb1e0h/AwtA01lHp8qwektWnV2nT2?= =?us-ascii?Q?qEMRHujnlN7BEjIEWCzkr63ctWLMxUYxVfjgT8j2aAZymiab6jjT9P+h7lwY?= =?us-ascii?Q?PW9nhMUwez2niB+Qcaf6e3Zpx3Dj1UASRPgMfPGfEvL24a+xDPKxU9oiwHDN?= =?us-ascii?Q?IuOnhXJifpasspouj1CDTsudHS33v4fov3eVR4SEesFDMJ7GyiUkoW0k6ejP?= =?us-ascii?Q?uSoT7r0uo023owy/4HUuKY8ajrFh+/gGlszOYyOBQJrgEhGtfpWIVifl43Eb?= =?us-ascii?Q?8Opb3U5n9AUFBtxfHSTxO1QJ6aqAbs+Z1DLghSgXvzhpuc6hVSlnAmjwAqjc?= =?us-ascii?Q?Hwxp1Riu+q5xo/V0wsvsNxps9Jck6z4O1b8L6x7JLppT1F0nwissnqBeO8xB?= =?us-ascii?Q?KRsTLwBPjM39YMcFor9B8aJGZ/aDFH6D3OjOlxA2cU3Fm+04drX//+yelmCu?= =?us-ascii?Q?8CFNLfyLKbIv5PhoaOdnod/kk97uvk2uZF54G3PSPVhdOFM90+zJ0Emwtu6B?= =?us-ascii?Q?R1dikZ8xdVLHuNZ7ZI3qUxfXJz+ksP+Z20s4Y7qvzDGC4Uoeh/9XFX4oqHSA?= =?us-ascii?Q?iNZ1GCgTSzC+AmyAjrQ5iQDrv5dQFgxtgOqYj+xWeOtSskiKEAxBE3D63fCM?= =?us-ascii?Q?TVgBQubIzvBuYaUsdojE2CuGqeuY0sUVy9tz6ECzFa17JWBCs6lpnunWrR9g?= =?us-ascii?Q?gpXdBf8OgYwSsfmNk+r3EAbc/mzj5AA3AjO5/9NLAs8am69MMOlJRhjIRZIr?= =?us-ascii?Q?XO75a03QCrkeRB2T/PIMXe99FTnCGIYQR17zgYbrzFgGuueD+BnD4pp20meC?= =?us-ascii?Q?W+B+Gr9uIQwN1SYdZAg5ikqdP/kHCAzlWxyx9g+CWblrtJhxEx3tRenLeJUF?= =?us-ascii?Q?P0/BgQvugFPl5IksHzPupblziR+QyBKzuR3co16YnNANKCdcC3OLL/E5Rl0O?= =?us-ascii?Q?lhJb83CUhk+QnOZyRHHj2P4U6DPw+WF7tJGCkBaQSxE2/T3UCkQkgrVwc9iF?= =?us-ascii?Q?r3v1DiF6hEOIHttnbMPuT2nD6TVUBrla1HYSQNNQ1qFLrMElh31uXX6a3hKb?= =?us-ascii?Q?3v/uFvTaUw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6bb8c9d6-30fb-40cd-ca87-08def94507cb X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2026 14:13:21.3833 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3Ny0w0refZrLGKp/rv5f0dI5kxHv9EmDkATVsaI0MfcZrcYyFwxv4ftOS0U49ONT X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8772 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260813_071329_770816_8EB27DA3 X-CRM114-Status: GOOD ( 15.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > Sorry I lost track of this thread and I just saw v4. > > In the mobile space, I haven't seen an SMMUv3 that does not support > RIL. Oh? That's very surprising, AFAIK none of our embedded chips support it yet.. Even the server chips are only just getting it. Are you sure? I gather it wasn't even available in ARM IP until recently ish? > However, I have seen workloads that are really sensitive to translation > latency (display, camera...). And I'd be concerned about those > regressing. Yes, those exist, but again, they are already facing these problems if running without RIL. And, for the common case of putting something into a carve out region it is not so likely even an expanded RIL will intersect with a reserved IOVA that has a high alignment. At least the things we have built are calibrated to handle a TLB reload occasionally. The isochronous TLB's are not even sized to be never-miss for all cases because things like 4k media require such a large amount of IOVA the area cost is too high. While others can do something else you are reaching into a pretty narrow condition to hit a problem: - HW that must have a never-miss TLB to work - HW that doesn't have a carve out, or has a badly aligned carve out - A SMMU that has RIL (non RIL is already worse) - A non-isochronos workload that regularly exceeds the RIL/single expansion thresholds - Unlucky IOVA allocation that places isochronous near other workloads in the IOVA space. > There is a clear trade-off here as you mentioned with TLBI latency, > would it be make sense to make that behviour configurable from a > module param? I think it makes sense for a driver to indicate to the core code that it needs isochronous and we can do more global things like change how single works as well. Having an isochronous flag on the domain, for example, would be a good overall direction. I'm inclined to leave this as is and let someone come with a specific problematic HW, rather that try to badly guess without much information if it might popssibly be a problem. Then we will know the HW and can mark the driver as I suggest above. Jason