From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4B62E8783A for ; Tue, 3 Feb 2026 14:34:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=DX8uB0OhR8lrxHTCcqmh2aIrK2vYNI+yqaFWxSs1AyQ=; b=VgwjjHl8E52JbB+2xswj+6//qs Y7/j0gTWd2GY5mi5oGxuFtUjVHPfXQXdI4aaVZq2FGZD3NcmnmlckiwEgqv6LBvCHkvqTE8vQqBGh TiGyJKyqB64ywfO5KUIKErPCdFV5pEva14AIUYBic9GIyIGZgwQb2r2zS2QWmQz5xHoNxPe2LfTYn DVvEMffvo5GgdoE9ZJg3R+i8p1ip9lt/u5owkT3Pnap8zHC7rhoNKfm8DqwpFJiGGRZSkao+hPRCg AJaGukwIwVo37rmn9Cb0XLeUw1jkJRDAAgdmlp/9FccSX1JESPkNMURf7SZ4DQyS0mxS3LD2VFowY B0K+2QDQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vnHTe-00000006m1i-0DNi; Tue, 03 Feb 2026 14:34:02 +0000 Received: from mail-westus3azon11010016.outbound.protection.outlook.com ([52.101.201.16] helo=PH7PR06CU001.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vnHTb-00000006m1B-1Odo for linux-arm-kernel@lists.infradead.org; Tue, 03 Feb 2026 14:34:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Us0RRVibIkCjrkOQstR9yw5hBxkNy+F9ZH5R/Ep1yU9GjjDzVmIhi7BCB82iIlXzM/ILkYisLMZB0P+wiZYA6yUA/Q7mvUerHO4sWi7dzdDdAltE8bBDrhsG1kGxPoO8kyw4m1rKmbY0pSCjs87U4YTen6Eud4cBbsstX2HAysJF+r64BQT1EbDNDVvARRfWXnZKSBU1poOOSESqvKWsjAnwDvMVdD+9xYiocFrCp7u+eW879zLS3LIHKSDSbtBVBDP0TZwFRr1nGxAuiJmCbAYNaNBqONCJbt7zUul+ZfnT+kRXadY+GqDba98FIHpoavfGq+IOecvz2LObEYIeDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DX8uB0OhR8lrxHTCcqmh2aIrK2vYNI+yqaFWxSs1AyQ=; b=V8oa8ysu3rJzky2jccX4nAR7A9rjqWdWo7/pDa8X0y62cOLH5MjFVKp6QWowJ6XQEjrhu6JbHjOeR3E9nHkKx/nXJGIeKnw1nHskToGmSqVUId6JgS2iw6xUnuPw8g1XLiUoX7zw3KGi26RZFShpPKGU00FlJHFfnwvAQ1qfQRL1xfUWtFOhuk7v9PP7AC1+HTwB86gjTFZgIwhATTnpfxDWrAnuXbkHHulb0X5NlVf+TMc8fYNwzZ85YtJPThRDcWNLo1Wl68lMfJHmaxoa+MROp8VrBj7NfFQ1+WRq84suovFEee6e38kzthTuzP3kC1bS2D3nyaoYRy12W/RzaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DX8uB0OhR8lrxHTCcqmh2aIrK2vYNI+yqaFWxSs1AyQ=; b=cVAsCpAITEhr1/kDx+yCFAn07LwXo3WX9iG9UQe8g5Ajc2AzaGB44Fm7aRwJ99FU12QCmCxHkqn40foIgDc7ItukA1MiCJW1xPQTdJCFJUrIps1SrM9T9rKYkUt6pqqJhslcIiEIS2OZoUfFrBKFqdf+gy28DNAvq29yd0DBjXpwrJuIlspRL1wV8jSYdi4z8ryvr0iH7gZhPTWZLTWJynThRP1UMSJOEjRacHwEfYh0gbEk4o2qImKsbzNvfrD3Ls09+s4298Eq5bStxCBg4BKKugvpiJ8teOSVCENXjYAsy6qwjs21nnMGWH1GdcEQxwJv9VOXyBXIEpGgXbjfDw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by DS0PR12MB7780.namprd12.prod.outlook.com (2603:10b6:8:152::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9587.12; Tue, 3 Feb 2026 14:33:49 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9587.010; Tue, 3 Feb 2026 14:33:49 +0000 Date: Tue, 3 Feb 2026 10:33:48 -0400 From: Jason Gunthorpe To: Nicolin Chen Cc: dan.j.williams@intel.com, "Tian, Kevin" , Jonathan Cameron , "will@kernel.org" , "robin.murphy@arm.com" , "bhelgaas@google.com" , "joro@8bytes.org" , "praan@google.com" , "baolu.lu@linux.intel.com" , "miko.lenczewski@arm.com" , "linux-arm-kernel@lists.infradead.org" , "iommu@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linux-cxl@vger.kernel.org" Subject: Re: [PATCH RFCv1 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices Message-ID: <20260203143348.GA3931454@nvidia.com> References: <20260121100307.00004e60@huawei.com> <20260121130315.GE1134360@nvidia.com> <6971b9406d069_1d33100df@dwillia2-mobl4.notmuch> <20260122131432.GJ1134360@nvidia.com> <69727e7ded712_3095100ab@dwillia2-mobl4.notmuch> <20260127150440.GF1134360@nvidia.com> <69795d0366a9_1d33100d3@dwillia2-mobl4.notmuch> <20260128130520.GV1134360@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BLAP220CA0019.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:32c::24) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|DS0PR12MB7780:EE_ X-MS-Office365-Filtering-Correlation-Id: 7eeeabc9-b6ba-4ac4-3469-08de63313ea0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?0ZIOWPr6Ogki/ex2qnL2fX3GDtgPFxWPYs8RwMsMEMPxHSoXN4eyZie8uFcD?= =?us-ascii?Q?x20iVd9Ap7Cj6kegPKsYcrDc/7usOJvIRz7gBl2lZ4BDJwszO2dNHXFyUNNa?= =?us-ascii?Q?rBN37b2X93ziwX/U20MQVROvMNJVxi+3MQAXQ+bb5+h0gYocJX0cyGzndNxp?= =?us-ascii?Q?osxOnmAVoL4nUQaUaUyO72MiC0SKu0PDW0VBg/AaW2g4j1/7BcwHkaDzu9ZQ?= =?us-ascii?Q?9q7tQiFXpMOzduav6evqLwCbMJZYl26A/zzpW7XfbI0CDxNy8dl6eqZvtUBH?= =?us-ascii?Q?U+U5v4K55f96CUpdJ0kZxeh4VqXAoMedSyVWe5YcE9qGSV+EDPBGaw3luoto?= =?us-ascii?Q?I7sRL1UgBS1dHnhdRJB9O0kwpLqq4PxnLScClXzhoOGNOD4Gahama1/DGX07?= =?us-ascii?Q?zUrJYJbbIJz6kLR0B4oAqaDGRrv7QQXI16SHjdfNJ1/XSh+5hkVwTDOS/vF4?= =?us-ascii?Q?zKYMjuVNlxXyGg9l0qRB+mtfyMqfAQy0O871RMOQjE2YF/CalLE9ENNsTS13?= =?us-ascii?Q?NYQTor0JDsqjwrVH0UmSXsEKcGOg58vaiYJvhK/WgLzE49rH+1hdnWmEfHkx?= =?us-ascii?Q?FPQ4Tuuz/FVt4hIfocPQPvhF/V+dGVqeTaweyDJouSshuSJbJOFSPN2Ex1L2?= =?us-ascii?Q?hjaMqsr+Hmi6ecVrjTfyTAxJZmifASAWlUWETBaflpG5wOr2Uv90BEhUj9p3?= =?us-ascii?Q?HF0KQFxn5r02l3xK5lvRAkDqsfz5MX6qO85qiivTNd0z0Rs4UuPjvZV9T5uJ?= =?us-ascii?Q?CwdtPTj0ieyDuEJMhcRr356Yq9NuuyFNUn3Z8cpCEnryNQRhW8MrxRGvHyfA?= =?us-ascii?Q?LI9ehtr1HPVVPz3SpHYBOtyQ0od+yi5YHvsIJlebJnF2n97vs0FN/fgtVo7S?= =?us-ascii?Q?3ZcqUn3hPNOod+MyjgI67HfAUjmPARqI9CPyyLc/Iv4OPUJ+PMo7eQojJJi+?= =?us-ascii?Q?HRhSZqrpcbPyXEvS/SwsMDOH0qFlblxS8m2nKw+4/gvZQpRPxQUy0KxuVGOJ?= =?us-ascii?Q?ps+FpNTpvERAKxM0XWAGIPr714fSmL0d+rBzBj70Xxd9L3YUenMqhK5cUXOT?= =?us-ascii?Q?2x38/0z3qxA1vGgqcHCamnEyIVDd/F+jdw+mllDRSy6smfBagqDzfITOeJeQ?= =?us-ascii?Q?tIryBtvtdwJ2tAx3k9McCVbMuhbtYVUOGkR6rb6SNQaJxZf7MMNGx8K0oMuk?= =?us-ascii?Q?bgHPbcw4uhf8or1pMVbiO6qsaHtg8cPeT3CJXYZslPUlqWfFiwdOfT3/ug2s?= =?us-ascii?Q?mizI071+VlIKjeQaS5McHPEIdUEp1NgHrA18MXSg6h73Ck4wC42vBK5G+GCQ?= =?us-ascii?Q?sUz/f0VmpiEa+xivrLLE/wUO/qHX2lqy9Bmy4Sx2x9+HjYi29K1Iuz6Dn74c?= =?us-ascii?Q?9UbofiF0OF7jAmoKetYq6Rjqlo57iO/RCYed9IqlF14mEzPhJi2qggq9dAmv?= =?us-ascii?Q?rq2hbwyTMvjlqW6u5sSFhqq996aCYJ0Ljc56ZYFj9WA5rzMSQjgyjNsxS7Wr?= =?us-ascii?Q?uLThssmQKwENKUwKdkqrcV24stwU7smcXmc7h8sZzS0d8CTgW21mfE+s4L9X?= =?us-ascii?Q?2jz+vBmaXzDEK/xWJcM=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ff5QlxBCs8ldZ59u1SA4g2Y4YV5uu7dmJzf7/RVh1rUE1rCGpXjn48R7g3rW?= =?us-ascii?Q?1u32MV08eZffHi5c1PTp7t1n+khD8Idi0nRsMQdQ3JJZTox1s2SNysEmkUMg?= =?us-ascii?Q?dqD8s4bSv8OalP5Ztos0qkiSLQj/FHHV1ZUmwglfVDNm8PS29qdtzBO1nFQ5?= =?us-ascii?Q?bAziDV7+SzoJMoUqJf5YdpSoLQMg0bHMoudaCNBwhjHzzwMZKWZWcEw/etwc?= =?us-ascii?Q?9v8Rqm3Qz73OqzyvY8+sX4iOkyOT+W7HABn8UneDzvHAprBk34mmfP1nJAs+?= =?us-ascii?Q?WC8/N9xfaNt7raitd+9Sln4ZSWjhiMyb+y2SrXLNucMp4o35Pe58VAjr6Wqv?= =?us-ascii?Q?kHEyBRkgC7dNN41RKW8zVMNp+QeqhZ5NohGVPNBfPXkoVy5R+y7pNVfWl0/C?= =?us-ascii?Q?K9VWr2QZR+liDc+/4waap/GxM42bhpdgdZN8MCxRmhC95Rk8GHbiiA+cv1Jz?= =?us-ascii?Q?mgkK9dmDe/iHnIoAt4viTmh/V2B34dZlCrpP3Q+6zqyFGe1zKOJtlnEQNXgG?= =?us-ascii?Q?CGnWEzfqrQWxj5PbZqKfZPTmFJ/gLdexa+VKx3iAGWdA5indYuzYOZSSJmRf?= =?us-ascii?Q?5suePRXlAls2vcarD4CXnukGSoHIIydUWispO7hSpauCWE6eemibm+Awl2Va?= =?us-ascii?Q?Gzt+fyd0UlvR6c09/loCicKt6r4vHZ+bJ475N6j7IbrY6xEh2pE5GPhU0GVY?= =?us-ascii?Q?MaFoVgVBCPowXgD3eykzspN/uxy5TiOamGYPfSBNDjV89UQptctYy7E/SwU6?= =?us-ascii?Q?OZ8Z2EbFzjJcjjBpIGWD87Iy99DOywpjxVAJIBuVbKpQNWZGzzGNgMxFDFGK?= =?us-ascii?Q?IlbbIRnMQleLgbwc9oQiRAjaxU3j8kw4b6DBgIRel7BOKnxZAYYAw7HU9lBP?= =?us-ascii?Q?d+uVSKiTvZxDuGSfcgh5bBzROKZK3ZZb9YWkWvVO/xEa8HSe/at2/sJvE/FW?= =?us-ascii?Q?WlYozQ5vrIgwhpeVhV7H4UoqgvG8Aqhmx5rZf3KuZy+sNtLmqZ4bnpFvbgsD?= =?us-ascii?Q?ln23UcAoSnb95q5E9e/88RsO6zsbMhnAZaq850pm5l6RC6/qL3ghwGbnhgiN?= =?us-ascii?Q?6QqgkvlZ9Pjt9L22h+r0AB1oopqnRhg1LihVPSRHCpEFVAetOG2OjN78WeWU?= =?us-ascii?Q?C6lln9Du0mCl6i9qqqq5qX/gP2gbqQg98Ej6oXPX/sweIqQ86iQ1SWLd2VsY?= =?us-ascii?Q?hnfuG/nbvUtGlhGhe1v87KxbHsGmTOn/z3pkurXo2eD8QhMNcbVdsccd2ZCG?= =?us-ascii?Q?q21Gra0F/hp2libiOSGyafwwGYCVveNlFSiWu496Fy6K3P3noto1j9qIZMSE?= =?us-ascii?Q?9UD/im0JwomODHq8fhPdzxx3wPuUBv/8xvjzZ+u+K2jupQPPTCaaU2ncNnWH?= =?us-ascii?Q?O+lhffgPEGN66MwSh/Sg/IWqO8em2albYMnByAWA0eY/1435Q6RnMI5DJk1g?= =?us-ascii?Q?1wRSImQzNQYyxQMqAKfL5wT3lAwL0pDlCiRdvOwCkhE4G3R2Upsd49rsS3GM?= =?us-ascii?Q?EYf6CB5gOBGaUpV1k7/MAHyaPHnUvPLgrjrKLMYFyv84dnITzrcB7j3h8/2P?= =?us-ascii?Q?3LJrNJ+5450bHUQVZdAHVolaf/bYIusNMUPavDadoVN6uNONYjDTm0LnF/Xr?= =?us-ascii?Q?U3cO9rw9Jx+bjhluKFETTXCvy/OPAiqays41JLRezFwv1djzJkUbsL+SnT+u?= =?us-ascii?Q?w+Smu9HsuvVFOzGziN+yhmzZUwdxqUjkIqH7QnWXLUWUtwtH5a1tJxcxomqW?= =?us-ascii?Q?niueL6FYGQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7eeeabc9-b6ba-4ac4-3469-08de63313ea0 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2026 14:33:49.0756 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZT4HZTdYGStjloUWA+B6DUdCcirsLyyxiJ18lPGbNy5exXvLy0+OSL+gMQ6mHXLm X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7780 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260203_063359_396966_77EDEB99 X-CRM114-Status: GOOD ( 23.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Feb 02, 2026 at 09:13:50PM -0800, Nicolin Chen wrote: > On Wed, Jan 28, 2026 at 09:05:20AM -0400, Jason Gunthorpe wrote: > > On Tue, Jan 27, 2026 at 04:49:07PM -0800, dan.j.williams@intel.com wrote: > > > > Yes, ARM took the position that ATS should be left disabled for > > > > IDENTITY both because of SMMU constraints and also because it made > > > > some sense that you wouldn't want ATS overhead just to get a 1:1 > > > > translation. > > > > > > Does this mean that ARM already today does not enable ATS until driver > > > attach, or is incremental work needed for that capability? > > > > All of the iommu drivers setup an iommu translation and enable ATS > > before any driver is bound. > > > > We would need to do more work in the core to leave the translation > > blocked when there is no driver. I don't think it is that difficult > > Hmm, not sure if we could use group->domain=NULL as "blocked.. Definately not, we need to use a proper blocked domain. > @@ -437,8 +437,6 @@ static int driver_sysfs_add(struct device *dev) > { > int ret; > > - bus_notify(dev, BUS_NOTIFY_BIND_DRIVER); > - > ret = sysfs_create_link(&dev->driver->p->kobj, &dev->kobj, > kobject_name(&dev->kobj)); > if (ret) > @@ -638,10 +636,12 @@ static int really_probe(struct device *dev, const struct device_driver *drv) > if (ret) > goto pinctrl_bind_failed; > > + bus_notify(dev, BUS_NOTIFY_BIND_DRIVER); > + > if (dev->bus->dma_configure) { > ret = dev->bus->dma_configure(dev); > if (ret) > - goto pinctrl_bind_failed; > + goto bus_notify_bind_failed; > } We shouldn't need any of these? The dma_configure callback already gets into the iommu code to validate the domain and restrict VFIO, no further callbacks should be needed. When the iommu driver is probed to the device we can assume no driver is bound and immediately attach the blocked domain. Jason