From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E170CCF9F8 for ; Fri, 7 Nov 2025 18:54:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=S/dh4Pj1aNdIFYDPt+HIvviv3QRuW9QN0KOksoJbBj0=; b=CYtcT0cw5OMPhbPWtcYvauR0Ei gikGcD33LPMBDuJbF4d3yzZXD8FvvQCcymV3J2m7/Q+oUqrvI+snJTbsZEjf7ZoKDve9i5aXiCmIB ozCwXpNhDuUHZ9BwY/0lnpfyDIyfs0zIFaMX7QippXifJuW8o9orl4SphzcZvkGCJxRVcwfDz1rc0 H6yFPLtSrYbkKEW8o8mKhrqqOfq4YUU3ysfX/j+Wqt668gsMMfchZwc0QhMJJabzSIUVlGQepUGdL hIP29n3ulp81jFbggTrQ1X9cXsdwPT5sTwn73W+cQCIG0rSigVp+xuleh5f3YST9M70xLO0JkmE1J 0eqcqQVw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vHRbG-00000000aLj-2UjY; Fri, 07 Nov 2025 18:54:18 +0000 Received: from mail-southcentralusazon11011011.outbound.protection.outlook.com ([40.93.194.11] helo=SN4PR0501CU005.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vHRbD-00000000aJf-2PzR; Fri, 07 Nov 2025 18:54:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=e2N3WH3Q0s2xy0fx+Kz2QAI4tgteTrLMqdENhBnAFsbiWAq9197EjxSwCevXpbiQS8Bhyzq8EsJaYoGDAJ8pNPmJ2eP1AUOBsVy5Z8ePqlqogjRDAnXxxh0Gk4UsB6ngU00f9XvegXmDIwCvOD4C2460CQG+cnq/rnmwTGzQsQ4EPOmLeNVn++5O5HAKa937iwFUA93O70icFOjS9w0r4NgT2bdMXRhbkxfpmCaITVLtojuhVZYF0MvxyXI6zd9/s7MsNyH4rdPYOHrnxN0XPCnNu3dfxpyAHNvhOHPihWJDxhLUxMv5dUpIx8HJeWYLk1Z5LZaL8W8SsKSN7sYazg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S/dh4Pj1aNdIFYDPt+HIvviv3QRuW9QN0KOksoJbBj0=; b=e4ZiMs+OpcpZfgqaGXQf3efyuVftC9yTxEbvIJpmy2A7w0Lj10lMOgMRuCNXLa1M3KXWVMK51TixW9CAnqlp80ttfXc4V1v9SjxXBbO8EfJFj76uHix3YC4ihfir4/ukgZ19Q1tRX1XsyD8/8M5Fb7hkIyTa8237e/QWHBiu77hIzs0hatbIFg4O2RuVbMS0BN9npLwBVP5hQWzCF1QrC+H+yEUUGgxTFyFc2AFyPbxQDubW9o9UcVnrryBgzVvp+ESGTshg5Hna4np5kZHBASvh9xjLHMnO5ABe9o1AH1MySiaNNGj/V0XxCInaYXiW7bN5eNjLWVfdFwNz63p85Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S/dh4Pj1aNdIFYDPt+HIvviv3QRuW9QN0KOksoJbBj0=; b=WQL2pUBsLkpov3w1ovfs2JP0WCMRwnJzTu7cG3VGqBZmjFz0IYjl1DPeOp/prNjJTeYqmQ4wF6QqzW3jtuM6MtQixwEPo8D4C8xk7CUyfDU3FRmsmdl0wEJGvIsR12Y3KfzlUEWrx55Dt+oBx3WR4WJ5YzFKB7RZaDfebGhg5oH6lZd+EzYZWtWb35NkId5X4JHFBVcCHt0ZfQr10w/EvyhwBpSde9WVk7QPbaIsJuNRtStclFhE5rCIKUan5DL3ixZZdL1IzEDTg+SPIs3LpKCq/qkC4Qfs+I7UHnZZnTjvf4SUmEnejJDs+BPeRvkeHTR+b/qK8XmoiympC2nRkA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB3613.namprd12.prod.outlook.com (2603:10b6:208:c1::17) by DS4PR12MB9819.namprd12.prod.outlook.com (2603:10b6:8:2a0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.13; Fri, 7 Nov 2025 18:54:11 +0000 Received: from MN2PR12MB3613.namprd12.prod.outlook.com ([fe80::1b3b:64f5:9211:608b]) by MN2PR12MB3613.namprd12.prod.outlook.com ([fe80::1b3b:64f5:9211:608b%4]) with mapi id 15.20.9298.006; Fri, 7 Nov 2025 18:54:11 +0000 Date: Fri, 7 Nov 2025 14:54:09 -0400 From: Jason Gunthorpe To: Nicolin Chen Cc: "Tian, Kevin" , "joro@8bytes.org" , "suravee.suthikulpanit@amd.com" , "will@kernel.org" , "robin.murphy@arm.com" , "sven@kernel.org" , "j@jannau.net" , "jean-philippe@linaro.org" , "robin.clark@oss.qualcomm.com" , "dwmw2@infradead.org" , "baolu.lu@linux.intel.com" , "yong.wu@mediatek.com" , "matthias.bgg@gmail.com" , "angelogioacchino.delregno@collabora.com" , "tjeznach@rivosinc.com" , "pjw@kernel.org" , "palmer@dabbelt.com" , "aou@eecs.berkeley.edu" , "heiko@sntech.de" , "schnelle@linux.ibm.com" , "mjrosato@linux.ibm.com" , "wens@csie.org" , "jernej.skrabec@gmail.com" , "samuel@sholland.org" , "thierry.reding@gmail.com" , "jonathanh@nvidia.com" , "iommu@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "asahi@lists.linux.dev" , "linux-arm-kernel@lists.infradead.org" , "linux-arm-msm@vger.kernel.org" , "linux-mediatek@lists.infradead.org" , "linux-riscv@lists.infradead.org" , "linux-rockchip@lists.infradead.org" , "linux-s390@vger.kernel.org" , "linux-sunxi@lists.linux.dev" , "linux-tegra@vger.kernel.org" , "virtualization@lists.linux.dev" , "patches@lists.linux.dev" Subject: Re: [PATCH v1 02/20] iommu: Introduce a test_dev domain op and an internal helper Message-ID: <20251107185409.GA1932966@nvidia.com> References: <32ce256a2ece5d63e99d5858f953586859818ffc.1760312725.git.nicolinc@nvidia.com> <20251103185400.GA1537560@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BN9PR03CA0610.namprd03.prod.outlook.com (2603:10b6:408:106::15) To MN2PR12MB3613.namprd12.prod.outlook.com (2603:10b6:208:c1::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR12MB3613:EE_|DS4PR12MB9819:EE_ X-MS-Office365-Filtering-Correlation-Id: 3be6ae1a-3d60-41ec-075a-08de1e2f096d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?JsmpoF1IZYe5z1K7MbwGUP1Fk6PGBfsRyhSY0Iwv+r5KnCJklCDFcNVihyxh?= =?us-ascii?Q?bSh58i+iLI5hmqoq7PszQ360G0awh5yLR2oOXBSctT9mkilrZLgNpJPA4oQg?= =?us-ascii?Q?bjD52KZ205MGAxOn38ATqo4sL+atFnQ/vSBGa9BoAvWxQXDusn8ZHnUW/7Kw?= =?us-ascii?Q?Ko7skszwn+D6gpI1yLwNeQvbC4QjjFdZBXXFhTKAG4cIKKscGEYJAQKgw/pT?= =?us-ascii?Q?ygYUuGoc0KXnTr6qOhJuNRmjI66VU0i9QSUDRpEuD+sBaCFfflxQrbnjxWa3?= =?us-ascii?Q?sC3Mgnlrx2w27buUZxADU2wA59fP59CtJL5tv+2TUddff3JN0lUL5wTvDksV?= =?us-ascii?Q?CzFuYc15xJhnnlJI3Fdoe5d+0ugOEW8lU4VKOxxtdO+X3c6/7nJgaCU8Or6Z?= =?us-ascii?Q?WkDhPAsD14SADSrPAEQ22B+ZBmoLY7J4bL/Bg0Q2g36ePCwWIlB+etsakBqp?= =?us-ascii?Q?5XOYJFCmhM+93TjdEma8oB3bZoihqOjrOjcG1AD8uDNBPzUnWH+m9unSSp3X?= =?us-ascii?Q?T/kA3O96rgMFsig7aCmeoXYyJNWdvwrZCpoVHb+3NqFtkxpr1/gcV37OVOdF?= =?us-ascii?Q?PoyWFS2SAui7o2xEmbaFUXRmAxaTR6w6lXJ8LIXdCE20Db/Tsy3wnpLG8p+o?= =?us-ascii?Q?V/6eRArxgDSyvDTs159LNyzAZJ73RueOfWb3gYjF/T89a4jw9qKlGHjTcSmo?= =?us-ascii?Q?A62U64gBwIz0CESNehn6XM1tUdIny7DhdWqJiVy1En8I+491vv67SDVxhcgo?= =?us-ascii?Q?Qgh83Pq/RvkxxgdbE83B7eyOhVqH0mBTlUN8oU5uublDTXS8VzVTqS/RVGR/?= =?us-ascii?Q?R8I7zRHSRKWdC48HWYdyXHlMksd48JS2w68PLGrWJ1yJ7QtlE53TflRdmZbR?= =?us-ascii?Q?Viq9c+ErYeMh2y6nNSlWEziAeudJ/ZnDZTJQmqt9pmuwle+/SdfSmcrIF1ds?= =?us-ascii?Q?cySCEWlmffu0dCaD+vZ7w/TE4wYVZ/Ap8yHksJNivLXbzTFK8RJVsvIS/wnP?= =?us-ascii?Q?UWcqPz0XMYxkeWkpctdhV72k7jaW/NNfu2SDnSZyxt+x7ognoag+9KcvPrWh?= =?us-ascii?Q?FLXwtiyj0QeTBNS64Ts4UzgpyTIG40x4B129Swg5QapC2WWXt2nKRv0SaSs7?= =?us-ascii?Q?nEOY5UaJYOjAvKoSRqOvGlrTvvwvc1aorHAyUL1bT7Vm1lDuM+lyPtqEOoYo?= =?us-ascii?Q?cxH40hq4ad+uvmozdN32U690wDxEPBD47OPzaQwq5WwI/r1/Hv+93l9pST9O?= =?us-ascii?Q?mh4cY8ySacmVB3AwiTtZFjrs7m6iRwpDxq5ksDE/xDhiGqrsPaNY5XuG+cFY?= =?us-ascii?Q?xWeBC7OhuXtSvuR9NpUTchtmf+sDk2gMYF94PUqnBqwp0GAMAQt713rJCWPj?= =?us-ascii?Q?WEs/qAc7SRyYOevBetH9NzvCtpbblIhK5aCksr/ZP7Q1GQq9UwassZLa0GJS?= =?us-ascii?Q?PsHaF8W3Yp3ETnVcndeh6D7AZENI97DoEq2j+zfDEtjHDrrPLvVerw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB3613.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?iLFQKnT3JaLPBqvAihvjxWzScgisUOQwxQ/KPE3fMYGw5hRNTtngKS4+PFDH?= =?us-ascii?Q?g8ZsV+3Ft/ll77wCZefKfhQ7bj5/lxXi2/qzqn9cX+rdN0ptUUm0xPXnct1B?= =?us-ascii?Q?HBm8hdgIz+FaBiJm/nE7Q6lp4Rz+2mD2YlDxGzm7w2UiyvuolFogd7s5hkGS?= =?us-ascii?Q?pXRESehyLfccbxqtTwZmUqHeC1S9CjKqeL0pT/qSaNy4QxguVkigiqS7uvSw?= =?us-ascii?Q?PqDDzy2FR/baHO+uT1Tua2Q22CczAXWqvGKsHEE3EXbtXrHQFeW7xf4Lb1YE?= =?us-ascii?Q?sATiCWA9dkeOkhhtyAsxNSPeqCahptAxMG1/AsRVxLp+escmtQgjHo0dFBeA?= =?us-ascii?Q?6wDcmbPqxdkNzlwWloFVSG353pEmbpR34H/YmaSQM4HRj2Sh0od0GIpXYgSp?= =?us-ascii?Q?IiH//+W+U0pjkb00bIca8itAEubKplTzZIlXoolasrdcIiWdxrB/beV6T4Uk?= =?us-ascii?Q?ylzuSMC3okFLGdV/nipbYxFVZ6j1rLO2VYtQyekCXEfhKV89BslqIXoNp8Ch?= =?us-ascii?Q?B474LrZ4fsleDHo8G7txW5cAmRxN9rxygiS9qZqPwdgVvRcrV0zdzeIA0E4f?= =?us-ascii?Q?rwN/sZSoMZiLHeW7ctrM7gh45j6YmPJObBE6jV8wuzmNmYZdKmxkY0bG3A7/?= =?us-ascii?Q?OTC0xsVpBQU+zafcvoc3weo861sfwE3GN5XqRMLmYyviDmfsbXHq113mSmvY?= =?us-ascii?Q?2nxQ1BXIOmeGsaMI40RvU54h070kyf6a7EC38sYpGNR74UO5MqDpUZYqCsxI?= =?us-ascii?Q?e0x27JOWO5IxkCJe/T0Nil2xHgPkRoZBE6fBzGGAPEjl9jtT47bVXx0CkjJQ?= =?us-ascii?Q?nEEEdPfNFIWN8hvNxsrC4poZCuZmVJSquJO33WSKl68FKYzIIG/UmsozTxc3?= =?us-ascii?Q?a3gAanPlNklrQDvowBjeiueo9SCZ2NJBh4o5tZ/uanLOpljrU8Tkne+efHGO?= =?us-ascii?Q?zrpJNo4I0Plbih3SUjO+K3Q1PwNoezbCD850GX1RRImyIy/s4O+GxUILfuVW?= =?us-ascii?Q?sHQ8mvE2JHKZJFTLQjsHlIe2LvBNTqM8saR/2DEfcZMVxEEvHyL1uui6tpmC?= =?us-ascii?Q?ga0C1EXO3c2N5Bxdt8uHGLC8C1I50GZWwvzw2dhovMh1GQegVAABl2huvpCI?= =?us-ascii?Q?HBdWhYfOKOIFKH552jnAbhtkvKrnpPWt2VljTwgumh1tnfIEYaBg0sJ03p1G?= =?us-ascii?Q?SkTCF7fI5LQDSmG/kTRIVfJebSuLRIL8w7Rx2i1VZI4bkvNPojm5nbgQ3HG+?= =?us-ascii?Q?1tndk/P7vjF/X51HxOSdzhCOZY7JxLUHKlXmXTOBGReGIrf0TA82f0R1jsAq?= =?us-ascii?Q?i0y30tStNbwj1KU0PRIltiyCGctCo1bRDKAbjoIxM0DKJBXND4POKXgrOWgG?= =?us-ascii?Q?oMbE75azpYTumPV8GR/uojuPVZSN4+BZAlkvZZoctC5I4ppncpcLuhyy+Wvn?= =?us-ascii?Q?IToUy6AHKcPE9rfH8rqV7DlpOx3efMW1lFh4HrKlsMfUBJNfsbZndwegjz0e?= =?us-ascii?Q?BUljScZ49Tv3rz2ctL058c5h1ERGRosmcc++AC4/g1LATUWYPCzDOcl6knOO?= =?us-ascii?Q?yvgzFLz6UWMCXr6SP+o=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3be6ae1a-3d60-41ec-075a-08de1e2f096d X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3613.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2025 18:54:11.0328 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: HYgc59X6PYsGk5HDOYFcNkKFzndLuSDwzWyqrKlRAEQdsAWVVbmfC9FCwsUVmtOn X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR12MB9819 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20251107_105415_617424_CC3A5078 X-CRM114-Status: GOOD ( 23.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Nov 05, 2025 at 10:18:01AM -0800, Nicolin Chen wrote: > On Wed, Nov 05, 2025 at 06:57:31AM +0000, Tian, Kevin wrote: > > > From: Jason Gunthorpe > > > Sent: Tuesday, November 4, 2025 2:54 AM > > > > > > On Thu, Oct 30, 2025 at 12:43:59PM -0700, Nicolin Chen wrote: > > > > > > > FWIW, I am thinking of another design based on Jason's remarks: > > > > https://lore.kernel.org/linux-iommu/aQBopHFub8wyQh5C@Asurada- > > > Nvidia/ > > > > > > > > So, instead of core initiating the round trip between the blocking > > > > domain and group->domain, it forwards dev_reset_prepare/done to the > > > > driver where it does a low-level attachment that wouldn't fail: > > > > For SMMUv3, it's an STE update. > > > > For intel_iommu, it seems to be the context table update? > > > > > > Kevin, how bad do you think the UAPI issue is if we ignore it? > > > > > > > yeah probably better to leave it. I didn't see a clean way and the > > value didn't justify the complexity. > > > > Regarding to PF reset, it's a devastating operation while the vf user > > is operating the vf w/o any awareness. there must be certain > > coordination in userspace. otherwise nobody can recover the > > registers. Comparing to that, solving the domain attach problem > > is less important... > > If I capture these correctly, we should go with a -EBUSY version: > - Reject concurrent attachments during a device reset > - Skip reset for devices having sibling group devices > - Allow PF to stop IOMMU, ignoring VFs > ? > > That sounds pretty much like this v4: > https://lore.kernel.org/linux-iommu/0f6021b500c74db33af8118210dd7a2b2fd31b3c.1756682135.git.nicolinc@nvidia.com/ > by dropping the SRIOV concern. It seems like the simplest answer.. I'd ignore the VFs, I think it is already really weird/dangerous to be resetting the PF while VFs have drivers bound.. Not sure there is anything we can do to make this work better. Jason