From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECC56CD3427 for ; Fri, 1 May 2026 23:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XOW+3LD3hVmaVzm1BjhUO50sBN+CcYoYiT5kIe678Gs=; b=OSYda7w66mtSMiPxZaN853bvxX vaGTtxfZpeoVT+vENAAgvRgmYwjD3FdH8ORI0v0ugmMWPAja1bRBrAl4qWGBd6M9fLxnv2FsdxgDn HzRh8ybjmwAcfx8bM1mzud9etN99xmrkDhSBTuxdUa4qmHvprBnPP4tqIX6bitnUIRNX3AR9VC+Ua aUMAiANig/+bce5YN6rCi7Kbu0kRr95OrcVb32INWge5qRVnKbrpbUpIxj2g8yihqqAxwfSqrc9gY g0qqs5GaXHKHWanMfxYYakLoFCYLQ65tNwU0EXL7nNKKSNwXpyDckPbG1KAOG5m6rej8zayUNUxvH xmGf+JBA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1wIxZP-00000007r4S-0a0V; Fri, 01 May 2026 23:46:55 +0000 Received: from mail-eastusazlp170120007.outbound.protection.outlook.com ([2a01:111:f403:c101::7] helo=BL0PR03CU003.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1wIxZM-00000007r3r-04kx for linux-arm-kernel@lists.infradead.org; Fri, 01 May 2026 23:46:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jP0ztPxXA7qXMWrOI+VRADHgvcI43/iSTW6nXTBi54q1vfwCZoH7MElKXUaZvl/O5LdTOW352lPBiNS2xHjoA1Rfdo8gHVuUIpvTmnu7WT7iQEpbpEOt91ZHc/B6x+bbdzl03qdbvwqVuQVs6kvMNVuqUOPOa+4LQ537S2HLDey1zC+57qMDICoO+6iXuETMugZxmva8EsU8oua+eSnpWnuAyRWW/GdbH29pRArviCac8WBsh94fVohLsg0eYnV3ngdivP6bHaz0x1UL5jl2AOSncrukbMvsHZr5ASOC8iV7lyQZFuCpOxsoxtBYqafAT5NymZtIcBo6Z9y4Ke831w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XOW+3LD3hVmaVzm1BjhUO50sBN+CcYoYiT5kIe678Gs=; b=K6OHQTGSL1qCDmAQQLCZla1mlXFf+1Gb6XWdIdixfkPrSYNglnlJ0VOxodKY9XqsgzYoEY/MhLLQN12FvUZ8fY8/MpT9O7XFlikz0D2IKFX6BO8tY/mGpKL0dg5/tVOqtrY6gypAzeznFDzfepk7NjafWjbs6YOugZZAoqzwUsM/vXi9195kd+6J9WAT4NmZbFF5wKDTbw01Mc/X8iCqg0mxB4VCTY822u1eZ8ZLY4QacWkbkZc0Dz+1d0d0ZRViuh7/Bhkh2SiDYKzmYF2+Ej1WMbUElrnNR3DxA0Hp0bvXrrtQ+v36OJGxp7lzr+CCPU5BmLVbavGkuI6hOCkAsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XOW+3LD3hVmaVzm1BjhUO50sBN+CcYoYiT5kIe678Gs=; b=bn/J6+ms8QQwa5/ZxOikHQ1+zDaE+op2Ld+S8HzJiJDL5qkaOv7wAuV3AyjSK5Ikp9WHGMdUA23LL8oL5MUvvJo2qtTY5rBuCOKrbyBtHM1iilRr8aAB53Cv+tyoYqGifeCneuOx5c+asEujzQcQ5dOyJhyx7mVKgaF3znh+FFEdRgD7USiQBTxIm215ugNrNP/6/vcy3clorRaP+XY2QaxcyilZNPUC+6JqthBsgnjrMqM17wq7gAo3EtR+goqGEa9WA7PJ8vpynx8HKzxBOUm9WdyPYtYYkRKFOSiaFeZzHJQPObBkp3lf7gUcEp3wGQZ5qQi80LohW7AkSN+I3A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by DS0PR12MB7511.namprd12.prod.outlook.com (2603:10b6:8:139::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.23; Fri, 1 May 2026 23:46:44 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9870.022; Fri, 1 May 2026 23:46:44 +0000 Date: Fri, 1 May 2026 20:46:41 -0300 From: Jason Gunthorpe To: "Dan Williams (nvidia)" Cc: Nicolin Chen , will@kernel.org, robin.murphy@arm.com, bhelgaas@google.com, joro@8bytes.org, praan@google.com, baolu.lu@linux.intel.com, kevin.tian@intel.com, miko.lenczewski@arm.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, dan.j.williams@intel.com, jonathan.cameron@huawei.com, vsethi@nvidia.com, linux-cxl@vger.kernel.org, nirmoyd@nvidia.com Subject: Re: [PATCH v4 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices Message-ID: <20260501234641.GB1381708@nvidia.com> References: <69f3cc82926_3291a910039@djbw-dev.notmuch> <69f536ed263dd_3291a910017@djbw-dev.notmuch> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <69f536ed263dd_3291a910017@djbw-dev.notmuch> X-ClientProxiedBy: BYAPR05CA0029.namprd05.prod.outlook.com (2603:10b6:a03:c0::42) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|DS0PR12MB7511:EE_ X-MS-Office365-Filtering-Correlation-Id: beb77938-ca07-4a0f-0309-08dea7dbe68e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: hS2bgXcGDxf/kF7zL3sfzYG9anhY9Vt2jXN4+eaUL1jbULq3wN7ys/XbKg20Oe3EGZ6SuhLb0Bioggu/kByif5PRivhPxYOGSXtNuDcoCwf0rM7m/8KwCsl0m1UKTvy+P9pcnOVgpaRZP1tkX0m5DBm5dDxEvGKT3avUyvLee5LCIF2HeF+0B8weyZtVhsXzb8hwlAkofauuosHzyqHe6nzIhgKp0LlfTYxZ7hb7R7vxHKhTdJaK4uQDwpktcBqDa/TAIx9xzJPFJuAcH79/Bb4utCIL+oyK80Z23m9ItZRWJk4Nn1EB9ytxTksDrg/9w8hH8GF4CtnyNqd1fpZoPLgqr42simArpVydM8ULW2iHmHJ906Y5erQ719xHkgiJi9JcLDovMMfO/kYo8L9/UY5mLUdTcO+LOwYOWheg9FIpglvMgrKnZ+VKescYu7Xzb5Br/UO4oBMnWhFpcxf5Ei3qIPiDJV37AgQNUHlw2HSeU2i1nSk/ADJEvZKsBsRRLqxGoWyCzGRLPGX20V2+M8MfaX6N0AGRP7foIig6MtV2L49DthPe85n6pMh5RQuADpxPrEQV56XlpC2QCttZsTREPvMSi/zDTdjQAUiQh3SCqfoMgp7HRBg9OI+lAvc4Tpd3sTZ7tCJUHhPaV9OvL+wXWmiwYRLyfSCzJ52qsf2HXU7SC2FeZKVDs5O9Jbgj X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JHzNKDww8fbfTiSI1pTp+Mmbej/3EBeRSXlML8XpoD4w4WQpQ4xlPxssV4R5?= =?us-ascii?Q?G5qapwUgsHhVwqeD1d3q/rxNa7Fft/W/6h8zVzzgPpvW1oN2Ukdjwsf0lx09?= =?us-ascii?Q?ZCsDvjiFlT8uII5Gl3ODPkj5N9+zeodNrZNEr08lPpDAOnNIDHx5vjzIiIjk?= =?us-ascii?Q?BguBpPTkI2R0a0Ad+gJCKPQOhgMgCbvIpW9foShPJdnEGuLBToA3pap9pmps?= =?us-ascii?Q?ymtLzjkmPoEpy4vmewZmRhiEq+jYvoTNODOnk/72XEBb+sGU0k5uVGXCB2rS?= =?us-ascii?Q?4bnOkR+/hQtBeBsd0/Rv1fqbVMala7rf4C5bduy2CS+kl8xemhZP/TEBQCtG?= =?us-ascii?Q?gA/YAgLLy2600bifdbFbzO09GDa0kvYDz8fdJ4LVIkvT3Azps83rvjkdohhP?= =?us-ascii?Q?1IfqH8wluitigdlkpz1P9p+bkBVd6eHTAlkkgkZuUjEsnYbjuwKdCWlUEsEu?= =?us-ascii?Q?Hu4LHuXdtkO4k2TyVdzbO2/EyjQk7mlPsteqCitTXwaDvibQWpQiGEnYY5pP?= =?us-ascii?Q?fs0rVbWd8DjBlueHISxoRbTHVghgj8woXmUdomeeU8LRz+6y6AsX+SJJt+FZ?= =?us-ascii?Q?Jokj8D1DDIPL7xVeoCZ1v5FYySjutoEg2jemkdsV21oJyZnc2GyUbSxpT/9j?= =?us-ascii?Q?9Uf9Wa4Gc7Q2kXq+KTT7eICOEXIEH2s3Sl/bGJfk479P0ACuyroK3xMglecw?= =?us-ascii?Q?4nTyEZ2uAoMruG7w+LOHqiuriXYjEmBmAXvt6tnN43NW57tQLX2OivNW9Fz8?= =?us-ascii?Q?7rQiW312mhxJHfrP3R6k1V0BVK4dk4RcG1Gy7R5YyBR6VWCi5hvd08uf11FR?= =?us-ascii?Q?x2qBm45d046D13Qep0RBzmSi/blXLmkfogdS9Z33udHLovlLvUZbpK2nYo3z?= =?us-ascii?Q?nu7oI1Zl8Vdfq8q2Yb4nqokeB/HxeCJzzR5iOtNtwA2bFDJSMN996AvYynFR?= =?us-ascii?Q?Yo0+gDNrq4wm/5tNysmw4F+EP9Y7B4mPUuOtR6E4VYz6mMNS2+bVxbSObLY6?= =?us-ascii?Q?lEU++OlfgBQn1fI9qAGgAqWuQL6sHF/JgEO9lPAxcm4d4DgNmMsXmLUCQ9pZ?= =?us-ascii?Q?bkyKzBnIoWfRmwJKbTex13ygVr1bG0LL9ibcwMtwmWZiAQ1aRUihZNbbV3AJ?= =?us-ascii?Q?/oZJskSnUyr2GjA4r+p3unmgf3t9l7Htfg6p2ygZwO8yVJqYYMvd2si/Hpqi?= =?us-ascii?Q?/W0YT34tWIE0aEk73+3jBs0uwj7OYe11msXH8JJXBOYQu9OlwDdU2I8Gp1hS?= =?us-ascii?Q?8JI15SPP+uibV1kNx8lONur/6X9Kk+fOAd6wtkxor/lPRak+qC68/w9O5tkd?= =?us-ascii?Q?iJEWrMUbXrwqWpAxoIJF+Z3nbV1yxG84BH1V+GR/GNqxxNIlsrZeWAOY4yzw?= =?us-ascii?Q?yDbFzBzJyrQ90Lx92AFUyvkEWwbxMoxgaPEjsXZIoJnXFSjrULYo+VvPPZsk?= =?us-ascii?Q?MTKcVt3M28KRLLZBloMf/ZcgVcSINxwWTfYYFLq44pudaG0Y3B4/AiitabBi?= =?us-ascii?Q?gyxG29WBfO7d2uYe80VKLK+cjgQWc9NAKu5cRKgMyd4BKKSKNfx7wIIJ2kIb?= =?us-ascii?Q?gcE++xGHh6Zu+3m6C93s73AHqYIUD1khzQdU+sjTw3EHIXnYHZEu8hBJmXYb?= =?us-ascii?Q?nSOG2M3cRDWFaXPhuFP9eYXTvYeB7aYrNW5F3XBJKWIkqe2eGfvOQLInE6Y3?= =?us-ascii?Q?8ZXJjYcUB7lAJjpu0+2veOC3K78V9NO/oPqTzjjNch/W6Dtu?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: beb77938-ca07-4a0f-0309-08dea7dbe68e X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2026 23:46:44.3134 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yAW3b12MGineqXgNod5Q1XfQUYPCDZ05OZlESTx1zOKn2nNm4Ya4TDfSFlTkK4o9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7511 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260501_164652_091293_12BFCA68 X-CRM114-Status: GOOD ( 11.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, May 01, 2026 at 04:27:41PM -0700, Dan Williams (nvidia) wrote: > You appear to be confusing Cache_Capable and Cache_Enabled. > > "8.2.1.3.1 DVSEC Flex Bus Port Capability" != "8.2.1.3.3 DVSEC Flex Bus Port Status" > > Cache_Capable is only a capability. To check that the device has > actually trained the CXL.cache alternate protocol you need to look at > the status register. The capable is probably a reasonable choice here unless you are confident the status will never change after the device is first discovered? ATS is being set early in the boot sequence. It is pretty safe to be over eager with the ATS enablement, less safe to get it off when it needs to be on. Jason