From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 35BCEFF8868 for ; Mon, 27 Apr 2026 16:31:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JRYSJF6nrSH7Det3WPja4JzgUkDVpaSEI3RwsPkFyHI=; b=TfXfhp+J8tSpkMdipo+EzglpMB C/G3LPA+VdsGSjubkvwcOPJTPiW6FQGUeoDAg3AMmsLqmAzQf3AnpIU6NoCVxJfr+OUhebm52prz7 UETr8Vg1RF8XJq4qwhgMjstJBKO16V+XWF9WSVje+uokCtreKBBky2sQS+JQwMvb/3ylmjiZ6+oo/ bSkg6pufCk+Rwrkd8eQIIlBvRkN7xadIs+SFrHhRATWO5QwAfVGW0V1869qzV6rHK3o+zb2GFsTul ZsOnYsOC6y8bIfV8JDmS1/Atcy18bqKPii+O44Z9E2GNGVaSJt5vz+yHSaXuSHVczlkllKc82oZSi inBFmTZA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1wHOs7-0000000HMsW-05ba; Mon, 27 Apr 2026 16:31:47 +0000 Received: from mgamail.intel.com ([198.175.65.21]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1wHOs4-0000000HMrz-16zb for linux-arm-kernel@lists.infradead.org; Mon, 27 Apr 2026 16:31:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1777307505; x=1808843505; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=RJ9UPp8EF7z1uvB7R2RYJWNX1kg1lSd61NxnC8m0o8k=; b=HB8wshe5xaesE8hiN1moXKOsa8sth351hJ2iPGrRRFYXKqH7a+W3K0QU z97s/k1pBLvDptCxtcCdmIohQhk+ABWl7R2la0r6qEKuyxzypj4ymaFNI keIlv+nbanNnfQ6lbqm3d0chtWfEPUR9NkIBzuwUpb3yr6+9JNs70O7F7 yrqewYg2FmUGcxcjgNixvsKZ7ChQgiOOWfcAL/55mpe6sTPWmNzlhrBTX XWUD3HgQCOyAuMUqdNXoeuojATWTaxVYvONTuOqwXek8BwuMWnj6Wg5gV QKZ1r06UjbwJG8HgRmmPR1SO3lIpx+dHzDEHAXnRd93sMpXjm6DgURFP3 w==; X-CSE-ConnectionGUID: l+/TwdeUTKCSoflisPPT+Q== X-CSE-MsgGUID: 7Cfj7Y3ERAmU3uo3vrALQA== X-IronPort-AV: E=McAfee;i="6800,10657,11769"; a="78093620" X-IronPort-AV: E=Sophos;i="6.23,202,1770624000"; d="scan'208";a="78093620" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa113.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2026 09:31:43 -0700 X-CSE-ConnectionGUID: LCC0RagbSjCpVrwYJkeKiQ== X-CSE-MsgGUID: D8PgNkBgQgSIF6wwRjcL7w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,202,1770624000"; d="scan'208";a="271816236" Received: from msatwood-mobl.amr.corp.intel.com (HELO [10.125.108.172]) ([10.125.108.172]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2026 09:31:42 -0700 Message-ID: <2efed1ea-e373-41d0-bc37-4adac8378169@intel.com> Date: Mon, 27 Apr 2026 09:31:39 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices To: Nicolin Chen , jgg@nvidia.com, will@kernel.org, robin.murphy@arm.com, bhelgaas@google.com Cc: joro@8bytes.org, praan@google.com, baolu.lu@linux.intel.com, kevin.tian@intel.com, miko.lenczewski@arm.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, dan.j.williams@intel.com, jonathan.cameron@huawei.com, vsethi@nvidia.com, linux-cxl@vger.kernel.org, nirmoyd@nvidia.com References: Content-Language: en-US From: Dave Jiang In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260427_093144_352897_84381151 X-CRM114-Status: GOOD ( 26.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 4/26/26 10:54 PM, Nicolin Chen wrote: > Controlled by the IOMMU driver, ATS is usually enabled "on demand" when a > given PASID on a device is attached to an I/O page table. This is working > even when a device has no translation on its RID (i.e., the RID is IOMMU > bypassed). > > However, certain PCIe devices require non-PASID ATS on their RID even when > the RID is IOMMU bypassed. Call this "always on". > > For example, CXL spec r4.0 notes in sec 3.2.5.13 Memory Type on CXL.cache: > "To source requests on CXL.cache, devices need to get the Host Physical > Address (HPA) from the Host by means of an ATS request on CXL.io." > > In other words, the CXL.cache capability requires ATS; otherwise, it can't > access host physical memory. > > Introduce a new pci_ats_always_on() helper for the IOMMU driver to scan a > PCI device and shift ATS policies between "on demand" and "always on". > > Add the support for CXL.cache devices first. Pre-CXL devices will be added > in quirks.c file. > > Note that pci_ats_always_on() validates against pci_ats_supported(), so we > ensure that untrusted devices (e.g. external ports) will not be always on. > This maintains the existing ATS security policy regarding potential side- > channel attacks via ATS. > > Cc: linux-cxl@vger.kernel.org > Suggested-by: Vikram Sethi > Suggested-by: Jason Gunthorpe > Reviewed-by: Jonathan Cameron > Reviewed-by: Jason Gunthorpe > Reviewed-by: Kevin Tian > Tested-by: Nirmoy Das > Acked-by: Nirmoy Das > Signed-off-by: Nicolin Chen Reviewed-by: Dave Jiang > --- > include/linux/pci-ats.h | 3 +++ > include/uapi/linux/pci_regs.h | 1 + > drivers/pci/ats.c | 43 +++++++++++++++++++++++++++++++++++ > 3 files changed, 47 insertions(+) > > diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h > index 75c6c86cf09dc..d14ba727d38b3 100644 > --- a/include/linux/pci-ats.h > +++ b/include/linux/pci-ats.h > @@ -12,6 +12,7 @@ int pci_prepare_ats(struct pci_dev *dev, int ps); > void pci_disable_ats(struct pci_dev *dev); > int pci_ats_queue_depth(struct pci_dev *dev); > int pci_ats_page_aligned(struct pci_dev *dev); > +bool pci_ats_always_on(struct pci_dev *dev); > #else /* CONFIG_PCI_ATS */ > static inline bool pci_ats_supported(struct pci_dev *d) > { return false; } > @@ -24,6 +25,8 @@ static inline int pci_ats_queue_depth(struct pci_dev *d) > { return -ENODEV; } > static inline int pci_ats_page_aligned(struct pci_dev *dev) > { return 0; } > +static inline bool pci_ats_always_on(struct pci_dev *dev) > +{ return false; } > #endif /* CONFIG_PCI_ATS */ > > #ifdef CONFIG_PCI_PRI > diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h > index 14f634ab9350d..6ac45be1008b8 100644 > --- a/include/uapi/linux/pci_regs.h > +++ b/include/uapi/linux/pci_regs.h > @@ -1349,6 +1349,7 @@ > /* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */ > #define PCI_DVSEC_CXL_DEVICE 0 > #define PCI_DVSEC_CXL_CAP 0xA > +#define PCI_DVSEC_CXL_CACHE_CAPABLE _BITUL(0) > #define PCI_DVSEC_CXL_MEM_CAPABLE _BITUL(2) > #define PCI_DVSEC_CXL_HDM_COUNT __GENMASK(5, 4) > #define PCI_DVSEC_CXL_CTRL 0xC > diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c > index ec6c8dbdc5e9c..fc871858b65bc 100644 > --- a/drivers/pci/ats.c > +++ b/drivers/pci/ats.c > @@ -205,6 +205,49 @@ int pci_ats_page_aligned(struct pci_dev *pdev) > return 0; > } > > +/* > + * CXL r4.0, sec 3.2.5.13 Memory Type on CXL.cache notes: to source requests on > + * CXL.cache, devices need to get the Host Physical Address (HPA) from the Host > + * by means of an ATS request on CXL.io. > + * > + * In other words, CXL.cache devices cannot access host physical memory without > + * ATS. > + */ > +static bool pci_cxl_ats_always_on(struct pci_dev *pdev) > +{ > + int offset; > + u16 cap; > + > + offset = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL, > + PCI_DVSEC_CXL_DEVICE); > + if (!offset) > + return false; > + > + if (pci_read_config_word(pdev, offset + PCI_DVSEC_CXL_CAP, &cap)) > + return false; > + > + return cap & PCI_DVSEC_CXL_CACHE_CAPABLE; > +} > + > +/** > + * pci_ats_always_on - Whether the PCI device requires ATS to be always enabled > + * @pdev: the PCI device > + * > + * Returns true, if the PCI device requires ATS for basic functional operation. > + */ > +bool pci_ats_always_on(struct pci_dev *pdev) > +{ > + if (pci_ats_disabled() || !pci_ats_supported(pdev)) > + return false; > + > + /* A VF inherits its PF's requirement for ATS function */ > + if (pdev->is_virtfn) > + pdev = pci_physfn(pdev); > + > + return pci_cxl_ats_always_on(pdev); > +} > +EXPORT_SYMBOL_GPL(pci_ats_always_on); > + > #ifdef CONFIG_PCI_PRI > void pci_pri_init(struct pci_dev *pdev) > {