From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0084BC43602 for ; Mon, 6 Jul 2026 19:46:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2oICNrQ1xwCDaEpKno9l01thkboCAbHyhE5Bxiua46I=; b=CHgHjlW/7m2xk0+OnQW4wGOH8F c3/punh6+a05ttvtcU/4wZgy2OuKFxZ88u1WspfgquBQWTTmMldma0FD9BVaZFmi/sjMVn9qakJB2 iZ4Piw4rJk+xG593GEFpLK2lKLc60Ezwpb1/vxYn92GHm+/pI6j+2qXi25Atga7ElSvut+rwPZ+oh 7vSLhzFh1/JxsMJuas/46TIrhc1zfkUDzue/yW3JiMgLL4AypNNUnLS1R1oICiQqwxXSjlMHZFGBs MkE50aDZ2mAId7kHpJVQjGFIOZ7J3289yTnMPt3FpZcU8h1eCufnwKlZIQlg7C5HYXW9r48+2rQBe tPSdqr9w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wgpGT-0000000DToR-1ehN; Mon, 06 Jul 2026 19:46:01 +0000 Received: from mail-westus3azon11012023.outbound.protection.outlook.com ([40.107.209.23] helo=PH8PR06CU001.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wgpGQ-0000000DTnv-2eI0 for linux-arm-kernel@lists.infradead.org; Mon, 06 Jul 2026 19:46:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PaumaG5Mzjr23BjTZO3d8wheOdOHnrI8j/Zpp0oQD4QNSqeN2iHpaW5CYceEklBVL/Kg3hKTal1pu8KfwSJXjsaPXGWwK4NhOTlqcy1DesKfokPaw7nU0HUzwiW5veuLbF+0QMMMJTNMWVR6JtJsvNs4S8GISOCgifDyTO+RcLa+g59OPFn4EbXblLpW/CtSW25W9MZ6yW7o/S8SYZuQgOo2gMEgLrF1iIhU83tbDNdbA1PraL9BnqGuSDck2tqS2ktqcxKudjf0/c+JFVTQq2qJgruP6sDEy/82N1ZACb6dN+gi2Z/gb4IRDNlfiBaZavf/O5QG3BInUk2ppMO9GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2oICNrQ1xwCDaEpKno9l01thkboCAbHyhE5Bxiua46I=; b=UNDdmsNqw5D7/TF6lFL9C/1o6YYLvxzlOFuluCeOjOPIPqGzWJLVlT9ZbsT+y8jFlydEnW5iNZjb9T7AL25Ze4C9fWAh4nkP8bIN5/M7iXi/va5rD2bnZY9+0JQNL1w+5YVhmuLrOQEXK9ziVvN6X9I7p5pulOE5Nn8P3oeWNkmho7UYvKnJLYL0mvUpTRJGAXO1J7G6+BVrQT/nsoDjt5xjNZ2rzWxBEvWQdgLWAC33uW16oR2NXzRKd8DD69XhUE6U1okKO2as+KY26SGS76V3njS9qS66UbLg0hbdaPpJwlJyarSXABOjpwH+F2oLdhYaM+A/KUFK1l1q0LEGOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2oICNrQ1xwCDaEpKno9l01thkboCAbHyhE5Bxiua46I=; b=h9WdVY9lBOR3E5jQcBvXUeJWXqpfVejS1wt1+QEa0tzSEkMEFqncnd1f9i6LcXFYPE4Lua4hzShIz6qR9aTS2HxpD+gNrknrERpeIHZGhRv5fz92bCj9OSy3/BGyWs6t4vYBIOStTdagwjQIFA6O1LpLdd7xkknKBBfU5Agd5rUc6GAxpUXSqQu2etTriBr4JfjjuP96Uof5A/KJ25pgWSIzqowT5gMZAAvIPx5ag9sAPAcpDvvYL7h8EHr390Vbsj+Jkx4qZNxd202DeVWVxRGKfegMULg0btJbUluiYBz4zkOrc7JkM1TxbtpYwZ1jUpGI2jsblibF8YlCAycm6Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by DS0PR12MB8248.namprd12.prod.outlook.com (2603:10b6:8:f3::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.181.10; Mon, 6 Jul 2026 19:45:51 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%4]) with mapi id 15.21.0181.008; Mon, 6 Jul 2026 19:45:51 +0000 Date: Mon, 6 Jul 2026 16:45:47 -0300 From: Jason Gunthorpe To: Robin Murphy Cc: iommu@lists.linux.dev, "Joerg Roedel (AMD)" , Jean-Philippe Brucker , linux-arm-kernel@lists.infradead.org, Will Deacon , David Matlack , Pasha Tatashin , patches@lists.linux.dev, Pranjal Shrivastava , Samiullah Khawaja , Mostafa Saleh Subject: Re: [PATCH v2 7/8] iommu/arm-smmu-v3: Change how the tlbi describes the invalidation Message-ID: <20260706194547.GC118313@nvidia.com> References: <7-v2-43074a57a53a+fb95-smmu_tlbi_jgg@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: YT4P288CA0005.CANP288.PROD.OUTLOOK.COM (2603:10b6:b01:d4::27) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|DS0PR12MB8248:EE_ X-MS-Office365-Filtering-Correlation-Id: 97079076-78ea-4a70-215f-08dedb972db5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|1800799024|366016|376014|18002099003|11063799006|22082099003|56012099006|4143699003; X-Microsoft-Antispam-Message-Info: hMeCR8GiJw5GmRuTMPt6DxtqMwqTe7z7DTnu52JruT7eX01eiQ9pj87klOO26X6KiAlqX1qDNNlDNaAeB2v32QL8muPvYdNcu6M6dv7xJITQKkzcQK8fHqphViTZQ8ggh/i0w/HcSszC6B8zM3ZT48hCnrD/yIfwkRYFzAWDVRSLYa8Fog3jFfU2kXvyCHv6DRbdMMGiJ0WFa33O/5QmVQ9QimENhLjvCBXzx6M0SBqAaqvxIuNGkQiwiWn9o5XJG21fRPuBi3GeffETwE4JXDDH8Bu8kcpTiOUf1OlAspTVRbUuL/v/5EZjjwz8nLuorDab4owb2VwJjTOaIZSNLWK808hPyP8faH1o7jKh1N0i8zMDWqdUsJVveFt4b7+CguEIUQPNdx+dV3YWuv/DWLsGROIudS6I598MVmjLOefQ1WBFVDKbF/AmxbDMVEDVMVuLRzm1NKtIWL0ouEmBVi6YMcxyYFp4Kvl/UOdUZgVPWyV4jnijsJryo98/7kH6yZ6pZDZjSh3Z1ncwK6Ma8lj18ivVX5LgREn/bWBzF/MDneyCpNBZCyKfTRvkrL8QKqu+Cp6ctoaz+xV5r/inm+0xgzaCiJppNdir2Oo5bE1LoD0uN3StAxs6SOL5rDUbskAps6WsOJD44z+eBUyONgxfMn8gtQt7w9vc5ElDufU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(1800799024)(366016)(376014)(18002099003)(11063799006)(22082099003)(56012099006)(4143699003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?EA4I11rfCvflLhq0LbaO9agKvxsnUMhhx6n4VgqJXssXZJHjQ78bX+ZR0XZP?= =?us-ascii?Q?2Anf+UXHdnawSsLONNPHvIiG0ID79AuDkI+8cCXTUYUc+o52nIqqwYcVy+r2?= =?us-ascii?Q?688igMIm64m5Zhb0J04bLUrx8IHXr2Dv7NA3Zg3Uqox5i8P6VI+5NVnbkAEL?= =?us-ascii?Q?ZF0onW3nV+EB+GsTKUBQSVr9oxwYuZNlBZYMx0T/+iI64Fk9nFKWGqOP937n?= =?us-ascii?Q?37bgtyqlQiXENSITze01KyUypLrHhTl4nE/mA+gkxZUizmu/oZgxcsx4bKWk?= =?us-ascii?Q?jPgZHjiQ8Or2OSeaX3XDLTMmpaeYiaaJ+3gectFnmz+mKiAiveU/jtKmlmVI?= =?us-ascii?Q?D7Z8eA2r18DZL6i5sFoz0Q0UktxaryIf1i2t8BfkXikR5+XFNL5/q2XwhgC6?= =?us-ascii?Q?db2wVrET6YPe8E6PesTdRNht82kHwFWynLC/P8JgDlQD/nnypGLf3/xFLxkP?= =?us-ascii?Q?dRujviSPJtGIiycjqsYnW6hzj5KU6KyezO6ec+nUrocqBJKZrp1bxG5UIDBQ?= =?us-ascii?Q?5emzjxtY2yoGvz86FM74Fjp2lWcUC8QuWLpEU5t/BiWXPh/tlnhzp17WgLUF?= =?us-ascii?Q?YDWV3r+zv/dPhfoOC5dq/S05sctyKQs0ib9M4/85CzvW1IGWNgdFfqyvB5Zt?= =?us-ascii?Q?t4UHRX4GIgTYJRFicepmHpswYxkc+4PlKpdZoExeAxQUTEeADZ9P8yl5dogc?= =?us-ascii?Q?S31BmQGirpC/idNJItOeXrGwdg1rBYD6pKD2p9O6O1im+ZM4eH7YFeU1zxc6?= =?us-ascii?Q?Keh1RmMFVEy8I+isN00YdVGKv3Xv0v4n+ueaW4U5v5s6T+DHkCB6oIPLifrr?= =?us-ascii?Q?wk4XxD+HVgFHmRFyudnGOyyk3cWL2gjPi5Mg/GF9pc884FMA/hmmzAnP2yDw?= =?us-ascii?Q?0AojL5VBfZlug7GvPUwO808xCOChhngUFUsM3m2ZoZ4VI6xHM3E2zavCXt3M?= =?us-ascii?Q?2C4/SfARh5M1Zg9HfBVGlB7Dswj8EBLeQlkMSVl7JldhDevA/0u3IDOVcJob?= =?us-ascii?Q?uxGYy0jyGzasmjbkfzEboViaYEVJNJD/nRgc/GAmxdRP480qrYO+XQVmsZQe?= =?us-ascii?Q?VWa1uADhwqvTRtm7EWs8o7drSbFmIdYqe8p5W0z9VTEKlW0mG+kSPbWADltR?= =?us-ascii?Q?IFHo3PBx511oAHBQ8UMCGyMf4qoSrzqwPLnctbLg8qRvaKwYwaVltIn3Fm5G?= =?us-ascii?Q?0IFKkpF8vMGAgJQzNTKZ8LPspUzK6WL+SvpPx9f4XBayuKvDAEtUcBw6D5hv?= =?us-ascii?Q?3nV7u9Q4/NpMLIiThz/goVYd08uiCnQPXOlKsDwPeW7xhAEkj70spKkU3OQx?= =?us-ascii?Q?tz5ceZQv+z2WeDU4XsBZGmHLwULS90yLCCKi+VjwJ3F/S/kFsU8vhtjxycMe?= =?us-ascii?Q?wjyqwUGMXdoFX30JfRM5q6DjlxBsNK6OXTAtet/9fkLlIyAhKp73MFy3vg9R?= =?us-ascii?Q?g/8xwqHt0aRcyYNxrTNDUlgwHO1uEQcnT5oDLmsEhd5gvNbHi/Z5f66I5mNO?= =?us-ascii?Q?jcV9Gpcl6jJ1gVzatdbTBeO5obOC1KUVCVI6NineDPbNGyXUCvYmTEnmW3ky?= =?us-ascii?Q?WogDE5L0JjWWo+egqueUvI7KHmPJc8AyPDNG/ev4M6DBmEyW9g17IT5SOFpJ?= =?us-ascii?Q?OQXJv6uHUQgV0j2SvH8RKD7WEJoTdrewv/uyephE7RjQF+37P29VcTWwEJq0?= =?us-ascii?Q?WO4aMBcYbuhcysFeslsULaVOUQKw/TPJxfK7MLhlk6+A9nkM?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97079076-78ea-4a70-215f-08dedb972db5 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2026 19:45:48.8422 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nKmb3dGNCA2p/DY7uU0F6kvc/dlzS/hY2eVecqkqyBeZ+dP40AqF9U/rvp8/rBws X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8248 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260706_124558_680298_E57CEAF7 X-CRM114-Status: GOOD ( 26.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Jul 06, 2026 at 07:00:15PM +0100, Robin Murphy wrote: > On 2026-07-06 5:26 pm, Jason Gunthorpe wrote: > [...] > > /* > > - * Generate a single range TLBI command covering [iova, iova+size). Sets > > + * Compute the TTL hint from leaf/table level bitmaps. 0 ttlt means no hint > > + * invalidate all levels. > > + */ > > +static unsigned int arm_smmu_compute_ttl(u8 leaf_bitmap, u8 table_bitmap, > > + unsigned int tg) > > +{ > > + int ttl; > > + > > + if (leaf_bitmap) { > > + if (is_power_of_2(leaf_bitmap)) > > + ttl = 3 - (int)__ffs(leaf_bitmap); > > + else > > + ttl = 0; > > + > > + if (table_bitmap) { > > + int table_ttl = 3 - (int)__ffs(table_bitmap) + 1; > > + > > + /* > > + * A RIL invalidation with !leaf_only clears out all > > + * table levels above the leaf level ttl only. > > + */ > > + if (table_ttl > ttl) > > + ttl = 0; > > + } > > + } else if (table_bitmap) { > > + ttl = 3 - (int)__ffs(table_bitmap) + 1; > > Maybe I'm misunderstanding what table_bitmap represents, but whichever way: > - if this case means purely changes to table (i.e. non-leaf) PTEs > themselves, then calculating any leaf level is pretty pointless. Yeah, it means this. RIL doesn't have a table-only mode, it always includes leaf invalidation. So my reasoning is the best RIL to form has a TTL which hints the fewest leaves, meaning the leaf level below the target tables. I guess 0/no-hint is the only other choice, do you think it is better? FWIW, a future feature I've talked about to remove unused tables would generate table only gathers, but currently it should be impossible. > > + /* 16K granule, ARM TTL=1 is reserved (SMMUv3 F.b Section 4.4.1) */ > > + if (tg == 14 && ttl == 1) > > + return 0; > > It's reserved in the absence of LPA2, i.e. when DS=0 (side note, please > refer to an up-to-date version of the architecture - F.b is pretty old by > now) Oh, I see the note is ment to be read that TTL=0b01 is legal when DS=1 and works normally.. > VA. If between the caller and the code above we can calculate that a block > entry exists where it cannot, then something is wrong and needs fixing > properly. Yes for leaves, but, the above is mixing tables into this as well, so at this point there can be ttl's for any point in the tree except the top most level, hence the check. If table-only changes to use TTL=0 then this could be a WARN_ON to detect malformed gathers. > > + /* ARM levels -1 and 0 cannot be hinted */ > > + if (ttl <= 0 || ttl > 3) > > + return 0; > > Similarly, no format allows blocks at level -1, so again if that check ever > did anything we'd already have bigger problems. In the remaining case, 4KB > with 52-bit VA *does* permit blocks at level 0, but it should hopefully be > obvious why that doesn't need special treatment here either... Yes, the -1 test can be WARN_ON as it is a malformed gather. The 0 level is just a comment than it isn't actually a hint anymore. I'll adjust them Thanks, Jason