[PATCH v6 0/3] Allow ATS to be always on for certain ATS-capable devices

[PATCH v6 0/3] Allow ATS to be always on for certain ATS-capable devices

[Nicolin Chen]

PCI ATS function is controlled by the IOMMU driver calling pci_enable_ats()
and pci_disable_ats() helpers. Depending on the driver implementation:
 - ATS should be enabled when a translation channel is enabled on a PASID
   (a typical SVA case).
 - ATS should be disabled when the device's RID is IOMMU bypassed and its
   PASIDs are not IOMMU-translated for any SVA use case.

However, certain PCIe devices require non-PASID ATS on the RID, even if the
RID is IOMMU bypassed. E.g. CXL.cache capability requires ATS to access the
physical memory; some pre-CXL NVIDIA GPUs also require the ATS to be always
on even when their RIDs are IOMMU bypassed.

Provide a helper function to detect CXL.cache capability and scan through a
pre-CXL device ID list.

As the initial use case, call the helper in ARM SMMUv3 driver and adapt the
driver accordingly with a per-device ats_always_on flag.

This is on Github:
https://github.com/nicolinc/iommufd/commits/pci_ats_always_on-v6

Changelog
v6
 * Add Acked-by from Bjorn
 * Slightly update commit message (Yi's comments)
 * [pci] Drop the redundant pci_ats_disabled() check
 * [smmu] Fix mismatched detach path against attach path
 * [smmu] Propagate arm_smmu_master_prepare_ats() error code
v5
 https://lore.kernel.org/all/cover.1779304390.git.nicolinc@nvidia.com/
 * Add Reviewed-by from Dave
 * Update comments in pci helpers
 * s/pci_ats_always_on/pci_ats_required
 * s/pci_cxl_ats_always_on/pci_cxl_ats_required
 * s/pci_dev_specific_ats_always_on/pci_dev_specific_ats_required
v4
 https://lore.kernel.org/all/cover.1777269009.git.nicolinc@nvidia.com/
 * Rebase on v7.1-rc1
 * Added Reviewed/Tested/Acked-by lines
 * Update commit messages and inline comments
 * [pci-quirks] Add range-based scan for NVIDIA GPUs
 * [smmu] Add missing arm_smmu_remove_master() in error path
 * [pci-ats] Don't init "cap=0"; check pci_read_config_word error
v3
 https://lore.kernel.org/all/cover.1772833963.git.nicolinc@nvidia.com/
 * Add Reviewed-by from Jonathan
 * Update function kdocs of PCI APIs
 * Simplify boolean return/variable computations
v2
 https://lore.kernel.org/all/cover.1771886695.git.nicolinc@nvidia.com/
 * s/non-CXL/pre-CXL
 * Rebase on v7.0-rc1
 * Update inline comments and commit message
 * Add WARN_ON back at !ptr in arm_smmu_clear_cd()
 * Add NVIDIA CX10 Family NVlink-C2C to the pre-CXL list
 * Do not add boolean parameter to arm_smmu_attach_dev_ste()
v1
 https://lore.kernel.org/all/cover.1768624180.git.nicolinc@nvidia.com/

Nicolin Chen (3):
  PCI: Add pci_ats_required() for CXL.cache capable devices
  PCI: Allow ATS to be always on for pre-CXL devices
  iommu/arm-smmu-v3: Allow ATS to be always on

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  1 +
 drivers/pci/pci.h                           |  9 +++
 include/linux/pci-ats.h                     |  3 +
 include/uapi/linux/pci_regs.h               |  1 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 81 ++++++++++++++++++---
 drivers/pci/ats.c                           | 47 ++++++++++++
 drivers/pci/quirks.c                        | 42 +++++++++++
 7 files changed, 175 insertions(+), 9 deletions(-)

-- 
2.43.0

[PATCH v6 1/3] PCI: Add pci_ats_required() for CXL.cache capable devices

[Nicolin Chen]

Controlled by IOMMU drivers, ATS can be enabled "on demand", when a given
PASID on a device is attached to an I/O page table. This is working, even
when a device has no translation on its RID (i.e., RID is IOMMU bypassed).

However, certain PCIe devices require non-PASID ATS on their RID even when
the RID is IOMMU bypassed. Call this "ATS always on" in IOMMU term.

For example, CXL spec r4.0 notes in sec 3.2.5.13 Memory Type on CXL.cache:
 "To source requests on CXL.cache, devices need to get the Host Physical
  Address (HPA) from the Host by means of an ATS request on CXL.io."

In other words, the CXL.cache capability requires ATS; otherwise, it can't
access host physical memory.

Introduce a new pci_ats_required() helper for the IOMMU driver to scan a
PCI device and shift ATS policies between "on demand" and "always on".

Add the support for CXL.cache devices first. Pre-CXL devices will be added
in quirks.c file.

Note that pci_ats_required() validates against pci_ats_supported(), so we
ensure that untrusted devices (e.g. external ports) will not be always on.
This maintains the existing ATS security policy regarding potential side-
channel attacks via ATS.

Cc: linux-cxl@vger.kernel.org
Suggested-by: Vikram Sethi <vsethi@nvidia.com>
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
Acked-by: Nirmoy Das <nirmoyd@nvidia.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 include/linux/pci-ats.h       |  3 +++
 include/uapi/linux/pci_regs.h |  1 +
 drivers/pci/ats.c             | 46 +++++++++++++++++++++++++++++++++++
 3 files changed, 50 insertions(+)

diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h
index 75c6c86cf09dc..f3723b6861294 100644
--- a/include/linux/pci-ats.h
+++ b/include/linux/pci-ats.h
@@ -12,6 +12,7 @@ int pci_prepare_ats(struct pci_dev *dev, int ps);
 void pci_disable_ats(struct pci_dev *dev);
 int pci_ats_queue_depth(struct pci_dev *dev);
 int pci_ats_page_aligned(struct pci_dev *dev);
+bool pci_ats_required(struct pci_dev *dev);
 #else /* CONFIG_PCI_ATS */
 static inline bool pci_ats_supported(struct pci_dev *d)
 { return false; }
@@ -24,6 +25,8 @@ static inline int pci_ats_queue_depth(struct pci_dev *d)
 { return -ENODEV; }
 static inline int pci_ats_page_aligned(struct pci_dev *dev)
 { return 0; }
+static inline bool pci_ats_required(struct pci_dev *dev)
+{ return false; }
 #endif /* CONFIG_PCI_ATS */
 
 #ifdef CONFIG_PCI_PRI
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 14f634ab9350d..6ac45be1008b8 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -1349,6 +1349,7 @@
 /* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */
 #define PCI_DVSEC_CXL_DEVICE				0
 #define  PCI_DVSEC_CXL_CAP				0xA
+#define   PCI_DVSEC_CXL_CACHE_CAPABLE			_BITUL(0)
 #define   PCI_DVSEC_CXL_MEM_CAPABLE			_BITUL(2)
 #define   PCI_DVSEC_CXL_HDM_COUNT			__GENMASK(5, 4)
 #define  PCI_DVSEC_CXL_CTRL				0xC
diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index ec6c8dbdc5e9c..84cd06d74fc9c 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -205,6 +205,52 @@ int pci_ats_page_aligned(struct pci_dev *pdev)
 	return 0;
 }
 
+/*
+ * CXL r4.0, sec 3.2.5.13 Memory Type on CXL.cache notes: to source requests on
+ * CXL.cache, devices need to get the Host Physical Address (HPA) from the Host
+ * by means of an ATS request on CXL.io.
+ *
+ * In other words, CXL.cache devices cannot access host physical memory without
+ * ATS.
+ *
+ * Check Cache_Capable instead of Cache_Enable because CXL.cache may be enabled
+ * after the caller uses this to make its ATS decision.
+ */
+static bool pci_cxl_ats_required(struct pci_dev *pdev)
+{
+	int offset;
+	u16 cap;
+
+	offset = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
+					   PCI_DVSEC_CXL_DEVICE);
+	if (!offset)
+		return false;
+
+	if (pci_read_config_word(pdev, offset + PCI_DVSEC_CXL_CAP, &cap))
+		return false;
+
+	return cap & PCI_DVSEC_CXL_CACHE_CAPABLE;
+}
+
+/**
+ * pci_ats_required - Whether the PCI device requires ATS
+ * @pdev: the PCI device
+ *
+ * Returns true, if the PCI device requires ATS for basic functional operation.
+ */
+bool pci_ats_required(struct pci_dev *pdev)
+{
+	if (!pci_ats_supported(pdev))
+		return false;
+
+	/* A VF inherits its PF's requirement for ATS function */
+	if (pdev->is_virtfn)
+		pdev = pci_physfn(pdev);
+
+	return pci_cxl_ats_required(pdev);
+}
+EXPORT_SYMBOL_GPL(pci_ats_required);
+
 #ifdef CONFIG_PCI_PRI
 void pci_pri_init(struct pci_dev *pdev)
 {
-- 
2.43.0

Re: [PATCH v6 1/3] PCI: Add pci_ats_required() for CXL.cache capable devices

[Bjorn Helgaas]

On Thu, May 21, 2026 at 01:34:20PM -0700, Nicolin Chen wrote:
> Controlled by IOMMU drivers, ATS can be enabled "on demand", when a given
> PASID on a device is attached to an I/O page table. This is working, even
> when a device has no translation on its RID (i.e., RID is IOMMU bypassed).
> 
> However, certain PCIe devices require non-PASID ATS on their RID even when
> the RID is IOMMU bypassed. Call this "ATS always on" in IOMMU term.
> 
> For example, CXL spec r4.0 notes in sec 3.2.5.13 Memory Type on CXL.cache:
>  "To source requests on CXL.cache, devices need to get the Host Physical
>   Address (HPA) from the Host by means of an ATS request on CXL.io."
> 
> In other words, the CXL.cache capability requires ATS; otherwise, it can't
> access host physical memory.
> 
> Introduce a new pci_ats_required() helper for the IOMMU driver to scan a
> PCI device and shift ATS policies between "on demand" and "always on".
> 
> Add the support for CXL.cache devices first. Pre-CXL devices will be added
> in quirks.c file.
> 
> Note that pci_ats_required() validates against pci_ats_supported(), so we
> ensure that untrusted devices (e.g. external ports) will not be always on.
> This maintains the existing ATS security policy regarding potential side-
> channel attacks via ATS.
> 
> Cc: linux-cxl@vger.kernel.org
> Suggested-by: Vikram Sethi <vsethi@nvidia.com>
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
> Acked-by: Nirmoy Das <nirmoyd@nvidia.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ...

> +bool pci_ats_required(struct pci_dev *pdev)
> +{
> +	if (!pci_ats_supported(pdev))
> +		return false;
> +
> +	/* A VF inherits its PF's requirement for ATS function */
> +	if (pdev->is_virtfn)
> +		pdev = pci_physfn(pdev);
> +
> +	return pci_cxl_ats_required(pdev);

I acked this before I saw this sashiko feedback, which looks like a
legit issue to me:

  Will this VF inheritance logic ever be reached?

  According to the PCIe SR-IOV specification (section 9.3.3.1), VFs do
  not implement the ATS Extended Capability, which means pdev->ats_cap
  is always 0 for VFs.

  Because of this, pci_ats_supported(pdev) will unconditionally return
  false for any VF. This causes the function to return false before it
  can ever reach the pdev->is_virtfn check.

  Could this prevent VFs from correctly enabling the ATS always on
  feature and leave them unable to access host memory without
  triggering IOMMU faults?

(From https://sashiko.dev/#/patchset/cover.1779304390.git.nicolinc%40nvidia.com)

I withdraw my ack for now until we figure out if it's a real issue.

> +}
> +EXPORT_SYMBOL_GPL(pci_ats_required);
> +
>  #ifdef CONFIG_PCI_PRI
>  void pci_pri_init(struct pci_dev *pdev)
>  {
> -- 
> 2.43.0
> 

Re: [PATCH v6 1/3] PCI: Add pci_ats_required() for CXL.cache capable devices

[Nicolin Chen]

On Thu, May 21, 2026 at 03:57:23PM -0500, Bjorn Helgaas wrote:
> On Thu, May 21, 2026 at 01:34:20PM -0700, Nicolin Chen wrote:
> > +bool pci_ats_required(struct pci_dev *pdev)
> > +{
> > +	if (!pci_ats_supported(pdev))
> > +		return false;
> > +
> > +	/* A VF inherits its PF's requirement for ATS function */
> > +	if (pdev->is_virtfn)
> > +		pdev = pci_physfn(pdev);
> > +
> > +	return pci_cxl_ats_required(pdev);
> 
> I acked this before I saw this sashiko feedback, which looks like a
> legit issue to me:
> 
>   Will this VF inheritance logic ever be reached?
> 
>   According to the PCIe SR-IOV specification (section 9.3.3.1), VFs do
>   not implement the ATS Extended Capability, which means pdev->ats_cap
>   is always 0 for VFs.
> 
>   Because of this, pci_ats_supported(pdev) will unconditionally return
>   false for any VF. This causes the function to return false before it
>   can ever reach the pdev->is_virtfn check.
> 
>   Could this prevent VFs from correctly enabling the ATS always on
>   feature and leave them unable to access host memory without
>   triggering IOMMU faults?
> 
> (From https://sashiko.dev/#/patchset/cover.1779304390.git.nicolinc%40nvidia.com)
>
> I withdraw my ack for now until we figure out if it's a real issue.

I did a bit of research here.

The existing pci_enable_ats() checks:

	if (!pci_ats_supported(dev))
		return -EINVAL;

at the top, prior to:

	if (dev->is_virtfn) {
		pdev = pci_physfn(dev);
		if (pdev->ats_stu != ps)
			return -EINVAL;

So, VF must *support* ATS (its ats_cap must be !0) so as to turn
on ATS, instead of relying on the PF's ats_cap.

IOW, Sashiko's comment "which means pdev->ats_cap is always 0 for
VFs" is wrong.

So, I kept the same order in this new pci_ats_required().

FWIW, I also did an inverted version and asked Sashiko to review:
====================================================================
> @@ -205,6 +205,52 @@ int pci_ats_page_aligned(struct pci_dev *pdev)
[ ... ]
> +bool pci_ats_required(struct pci_dev *pdev)
> +{
> +	/* A VF inherits its PF's requirement for ATS function */
> +	if (pdev->is_virtfn)
> +		pdev = pci_physfn(pdev);
> +
> +        if (!pci_ats_supported(pdev))
> +                return false;

This isn't a bug, but it looks like these two lines are indented with spaces
instead of tabs.
Also, since pdev is reassigned to pci_physfn(pdev) before checking ATS
support, does this incorrectly check the ATS capability of the PF rather than
the VF?
If a VF has its ATS support explicitly disabled (for example, via PCIe quirks
that set vf->ats_cap = 0), this check might bypass the VF's specific
properties and incorrectly return true based on the PF's capabilities.
Consequently, if a caller attempts to enable ATS on the VF by calling
pci_enable_ats(), it will fail because pci_enable_ats() correctly checks
pci_ats_supported() against the VF.
Would it be better to evaluate pci_ats_supported() on the original pdev
before reassigning it to the PF?
====================================================================

What would you like me to do, Bjorn?

Nicolin

Re: [PATCH v6 1/3] PCI: Add pci_ats_required() for CXL.cache capable devices

[Bjorn Helgaas]

On Thu, May 21, 2026 at 02:07:34PM -0700, Nicolin Chen wrote:
> On Thu, May 21, 2026 at 03:57:23PM -0500, Bjorn Helgaas wrote:
> > On Thu, May 21, 2026 at 01:34:20PM -0700, Nicolin Chen wrote:
> > > +bool pci_ats_required(struct pci_dev *pdev)
> > > +{
> > > +	if (!pci_ats_supported(pdev))
> > > +		return false;
> > > +
> > > +	/* A VF inherits its PF's requirement for ATS function */
> > > +	if (pdev->is_virtfn)
> > > +		pdev = pci_physfn(pdev);
> > > +
> > > +	return pci_cxl_ats_required(pdev);
> > 
> > I acked this before I saw this sashiko feedback, which looks like a
> > legit issue to me:
> > 
> >   Will this VF inheritance logic ever be reached?
> > 
> >   According to the PCIe SR-IOV specification (section 9.3.3.1), VFs do
> >   not implement the ATS Extended Capability, which means pdev->ats_cap
> >   is always 0 for VFs.

Huh.  I wish sashiko would include the spec revision because that sure
looks wrong.  In PCIe r7.0, there is no sec 9.3.3.1.  In PCIe r6.0,
sec 9.3.3.1 is the SR-IOV Extended Capability, which doesn't mention
ATS.  In both, sec 10.5.1 is the ATS Extended Capability and says both
PFs and VFs can implement it.

So I think this is OK as-is:

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

Re: [PATCH v6 1/3] PCI: Add pci_ats_required() for CXL.cache capable devices

[Nicolin Chen]

On Thu, May 21, 2026 at 04:31:23PM -0500, Bjorn Helgaas wrote:
> On Thu, May 21, 2026 at 02:07:34PM -0700, Nicolin Chen wrote:
> > On Thu, May 21, 2026 at 03:57:23PM -0500, Bjorn Helgaas wrote:
> > > On Thu, May 21, 2026 at 01:34:20PM -0700, Nicolin Chen wrote:
> > > > +bool pci_ats_required(struct pci_dev *pdev)
> > > > +{
> > > > +	if (!pci_ats_supported(pdev))
> > > > +		return false;
> > > > +
> > > > +	/* A VF inherits its PF's requirement for ATS function */
> > > > +	if (pdev->is_virtfn)
> > > > +		pdev = pci_physfn(pdev);
> > > > +
> > > > +	return pci_cxl_ats_required(pdev);
> > > 
> > > I acked this before I saw this sashiko feedback, which looks like a
> > > legit issue to me:
> > > 
> > >   Will this VF inheritance logic ever be reached?
> > > 
> > >   According to the PCIe SR-IOV specification (section 9.3.3.1), VFs do
> > >   not implement the ATS Extended Capability, which means pdev->ats_cap
> > >   is always 0 for VFs.
> 
> Huh.  I wish sashiko would include the spec revision because that sure
> looks wrong.  In PCIe r7.0, there is no sec 9.3.3.1.  In PCIe r6.0,
> sec 9.3.3.1 is the SR-IOV Extended Capability, which doesn't mention
> ATS.  In both, sec 10.5.1 is the ATS Extended Capability and says both
> PFs and VFs can implement it.

I am glad you checked the spec. I should have done the same.

> So I think this is OK as-is:
> 
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>

Thanks!
Nicolin

Re: [PATCH v6 1/3] PCI: Add pci_ats_required() for CXL.cache capable devices

[Yi Liu]

On 5/22/26 04:34, Nicolin Chen wrote:
> Controlled by IOMMU drivers, ATS can be enabled "on demand", when a given
> PASID on a device is attached to an I/O page table. This is working, even
> when a device has no translation on its RID (i.e., RID is IOMMU bypassed).
> 
> However, certain PCIe devices require non-PASID ATS on their RID even when
> the RID is IOMMU bypassed. Call this "ATS always on" in IOMMU term.
> 
> For example, CXL spec r4.0 notes in sec 3.2.5.13 Memory Type on CXL.cache:
>   "To source requests on CXL.cache, devices need to get the Host Physical
>    Address (HPA) from the Host by means of an ATS request on CXL.io."
> 
> In other words, the CXL.cache capability requires ATS; otherwise, it can't
> access host physical memory.
> 
> Introduce a new pci_ats_required() helper for the IOMMU driver to scan a
> PCI device and shift ATS policies between "on demand" and "always on".
> 
> Add the support for CXL.cache devices first. Pre-CXL devices will be added
> in quirks.c file.
> 
> Note that pci_ats_required() validates against pci_ats_supported(), so we
> ensure that untrusted devices (e.g. external ports) will not be always on.
> This maintains the existing ATS security policy regarding potential side-
> channel attacks via ATS.
> 
> Cc: linux-cxl@vger.kernel.org
> Suggested-by: Vikram Sethi <vsethi@nvidia.com>
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
> Acked-by: Nirmoy Das <nirmoyd@nvidia.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ---

Reviewed-by: Yi Liu <yi.l.liu@intel.com>

>   include/linux/pci-ats.h       |  3 +++
>   include/uapi/linux/pci_regs.h |  1 +
>   drivers/pci/ats.c             | 46 +++++++++++++++++++++++++++++++++++
>   3 files changed, 50 insertions(+)
> 
> diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h
> index 75c6c86cf09dc..f3723b6861294 100644
> --- a/include/linux/pci-ats.h
> +++ b/include/linux/pci-ats.h
> @@ -12,6 +12,7 @@ int pci_prepare_ats(struct pci_dev *dev, int ps);
>   void pci_disable_ats(struct pci_dev *dev);
>   int pci_ats_queue_depth(struct pci_dev *dev);
>   int pci_ats_page_aligned(struct pci_dev *dev);
> +bool pci_ats_required(struct pci_dev *dev);
>   #else /* CONFIG_PCI_ATS */
>   static inline bool pci_ats_supported(struct pci_dev *d)
>   { return false; }
> @@ -24,6 +25,8 @@ static inline int pci_ats_queue_depth(struct pci_dev *d)
>   { return -ENODEV; }
>   static inline int pci_ats_page_aligned(struct pci_dev *dev)
>   { return 0; }
> +static inline bool pci_ats_required(struct pci_dev *dev)
> +{ return false; }
>   #endif /* CONFIG_PCI_ATS */
>   
>   #ifdef CONFIG_PCI_PRI
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index 14f634ab9350d..6ac45be1008b8 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -1349,6 +1349,7 @@
>   /* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */
>   #define PCI_DVSEC_CXL_DEVICE				0
>   #define  PCI_DVSEC_CXL_CAP				0xA
> +#define   PCI_DVSEC_CXL_CACHE_CAPABLE			_BITUL(0)
>   #define   PCI_DVSEC_CXL_MEM_CAPABLE			_BITUL(2)
>   #define   PCI_DVSEC_CXL_HDM_COUNT			__GENMASK(5, 4)
>   #define  PCI_DVSEC_CXL_CTRL				0xC
> diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
> index ec6c8dbdc5e9c..84cd06d74fc9c 100644
> --- a/drivers/pci/ats.c
> +++ b/drivers/pci/ats.c
> @@ -205,6 +205,52 @@ int pci_ats_page_aligned(struct pci_dev *pdev)
>   	return 0;
>   }
>   
> +/*
> + * CXL r4.0, sec 3.2.5.13 Memory Type on CXL.cache notes: to source requests on
> + * CXL.cache, devices need to get the Host Physical Address (HPA) from the Host
> + * by means of an ATS request on CXL.io.
> + *
> + * In other words, CXL.cache devices cannot access host physical memory without
> + * ATS.
> + *
> + * Check Cache_Capable instead of Cache_Enable because CXL.cache may be enabled
> + * after the caller uses this to make its ATS decision.
> + */
> +static bool pci_cxl_ats_required(struct pci_dev *pdev)
> +{
> +	int offset;
> +	u16 cap;
> +
> +	offset = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
> +					   PCI_DVSEC_CXL_DEVICE);
> +	if (!offset)
> +		return false;
> +
> +	if (pci_read_config_word(pdev, offset + PCI_DVSEC_CXL_CAP, &cap))
> +		return false;
> +
> +	return cap & PCI_DVSEC_CXL_CACHE_CAPABLE;
> +}
> +
> +/**
> + * pci_ats_required - Whether the PCI device requires ATS
> + * @pdev: the PCI device
> + *
> + * Returns true, if the PCI device requires ATS for basic functional operation.
> + */
> +bool pci_ats_required(struct pci_dev *pdev)
> +{
> +	if (!pci_ats_supported(pdev))
> +		return false;
> +
> +	/* A VF inherits its PF's requirement for ATS function */
> +	if (pdev->is_virtfn)
> +		pdev = pci_physfn(pdev);
> +
> +	return pci_cxl_ats_required(pdev);
> +}
> +EXPORT_SYMBOL_GPL(pci_ats_required);
> +
>   #ifdef CONFIG_PCI_PRI
>   void pci_pri_init(struct pci_dev *pdev)
>   {

[PATCH v6 2/3] PCI: Allow ATS to be always on for pre-CXL devices

[Nicolin Chen]

Some NVIDIA GPU/NIC devices, though they don't implement CXL config space,
have many CXL-like properties. Call this kind "pre-CXL".

Similar to CXL.cache capability, these pre-CXL devices also require the ATS
function even when their RIDs are IOMMU bypassed, i.e. keep ATS "always on"
v.s. "on demand" when a non-zero PASID line gets enabled in SVA use cases.

Introduce pci_dev_specific_ats_required() quirk function to scan a list of
IDs for these devices. Then, include it in pci_ats_required().

Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Nirmoy Das <nirmoyd@nvidia.com>
Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 drivers/pci/pci.h    |  9 +++++++++
 drivers/pci/ats.c    |  3 ++-
 drivers/pci/quirks.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 4a14f88e543a2..e8ad27abb1cfe 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -1155,6 +1155,15 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, bool probe)
 }
 #endif
 
+#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_PCI_ATS)
+bool pci_dev_specific_ats_required(struct pci_dev *dev);
+#else
+static inline bool pci_dev_specific_ats_required(struct pci_dev *dev)
+{
+	return false;
+}
+#endif
+
 #if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)
 int acpi_get_rc_resources(struct device *dev, const char *hid, u16 segment,
 			  struct resource *res);
diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index 84cd06d74fc9c..96efa00d97433 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -247,7 +247,8 @@ bool pci_ats_required(struct pci_dev *pdev)
 	if (pdev->is_virtfn)
 		pdev = pci_physfn(pdev);
 
-	return pci_cxl_ats_required(pdev);
+	return pci_cxl_ats_required(pdev) ||
+	       pci_dev_specific_ats_required(pdev);
 }
 EXPORT_SYMBOL_GPL(pci_ats_required);
 
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index caaed1a01dc02..c0242f3e9f063 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -5715,6 +5715,48 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459, quirk_intel_e2000_no_ats);
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a, quirk_intel_e2000_no_ats);
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c, quirk_intel_e2000_no_ats);
+
+static bool quirk_nvidia_gpu_ats_required(struct pci_dev *pdev)
+{
+	switch (pdev->device) {
+	case 0x2e00 ... 0x2e3f: /* GB20B */
+		return true;
+	}
+	return false;
+}
+
+static const struct pci_dev_ats_required {
+	u16 vendor;
+	u16 device;
+	bool (*ats_required)(struct pci_dev *dev);
+} pci_dev_ats_required[] = {
+	/* NVIDIA GPUs */
+	{ PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID, quirk_nvidia_gpu_ats_required },
+	/* NVIDIA CX10 Family NVlink-C2C */
+	{ PCI_VENDOR_ID_MELLANOX, 0x2101, NULL },
+	{ 0 }
+};
+
+/*
+ * Some NVIDIA devices do not implement CXL config space, but present as PCIe
+ * devices that can issue CXL-like cache operations like CXL.cache. Thus, they
+ * require ATS to obtain host physical addresses, like pci_cxl_ats_required().
+ */
+bool pci_dev_specific_ats_required(struct pci_dev *pdev)
+{
+	const struct pci_dev_ats_required *i;
+
+	for (i = pci_dev_ats_required; i->vendor; i++) {
+		if (i->vendor != pdev->vendor)
+			continue;
+		if (i->ats_required && i->ats_required(pdev))
+			return true;
+		if (!i->ats_required && i->device == pdev->device)
+			return true;
+	}
+
+	return false;
+}
 #endif /* CONFIG_PCI_ATS */
 
 /* Freescale PCIe doesn't support MSI in RC mode */
-- 
2.43.0

Re: [PATCH v6 2/3] PCI: Allow ATS to be always on for pre-CXL devices

[Yi Liu]

On 5/22/26 04:34, Nicolin Chen wrote:
> Some NVIDIA GPU/NIC devices, though they don't implement CXL config space,
> have many CXL-like properties. Call this kind "pre-CXL".
> 
> Similar to CXL.cache capability, these pre-CXL devices also require the ATS
> function even when their RIDs are IOMMU bypassed, i.e. keep ATS "always on"
> v.s. "on demand" when a non-zero PASID line gets enabled in SVA use cases.
> 
> Introduce pci_dev_specific_ats_required() quirk function to scan a list of
> IDs for these devices. Then, include it in pci_ats_required().
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Nirmoy Das <nirmoyd@nvidia.com>
> Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ---

Reviewed-by: Yi Liu <yi.l.liu@intel.com>

>   drivers/pci/pci.h    |  9 +++++++++
>   drivers/pci/ats.c    |  3 ++-
>   drivers/pci/quirks.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>   3 files changed, 53 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 4a14f88e543a2..e8ad27abb1cfe 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -1155,6 +1155,15 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, bool probe)
>   }
>   #endif
>   
> +#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_PCI_ATS)
> +bool pci_dev_specific_ats_required(struct pci_dev *dev);
> +#else
> +static inline bool pci_dev_specific_ats_required(struct pci_dev *dev)
> +{
> +	return false;
> +}
> +#endif
> +
>   #if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)
>   int acpi_get_rc_resources(struct device *dev, const char *hid, u16 segment,
>   			  struct resource *res);
> diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
> index 84cd06d74fc9c..96efa00d97433 100644
> --- a/drivers/pci/ats.c
> +++ b/drivers/pci/ats.c
> @@ -247,7 +247,8 @@ bool pci_ats_required(struct pci_dev *pdev)
>   	if (pdev->is_virtfn)
>   		pdev = pci_physfn(pdev);
>   
> -	return pci_cxl_ats_required(pdev);
> +	return pci_cxl_ats_required(pdev) ||
> +	       pci_dev_specific_ats_required(pdev);
>   }
>   EXPORT_SYMBOL_GPL(pci_ats_required);
>   
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index caaed1a01dc02..c0242f3e9f063 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -5715,6 +5715,48 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);
>   DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459, quirk_intel_e2000_no_ats);
>   DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a, quirk_intel_e2000_no_ats);
>   DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c, quirk_intel_e2000_no_ats);
> +
> +static bool quirk_nvidia_gpu_ats_required(struct pci_dev *pdev)
> +{
> +	switch (pdev->device) {
> +	case 0x2e00 ... 0x2e3f: /* GB20B */
> +		return true;
> +	}
> +	return false;
> +}
> +
> +static const struct pci_dev_ats_required {
> +	u16 vendor;
> +	u16 device;
> +	bool (*ats_required)(struct pci_dev *dev);
> +} pci_dev_ats_required[] = {
> +	/* NVIDIA GPUs */
> +	{ PCI_VENDOR_ID_NVIDIA, PCI_ANY_ID, quirk_nvidia_gpu_ats_required },
> +	/* NVIDIA CX10 Family NVlink-C2C */
> +	{ PCI_VENDOR_ID_MELLANOX, 0x2101, NULL },
> +	{ 0 }
> +};
> +
> +/*
> + * Some NVIDIA devices do not implement CXL config space, but present as PCIe
> + * devices that can issue CXL-like cache operations like CXL.cache. Thus, they
> + * require ATS to obtain host physical addresses, like pci_cxl_ats_required().
> + */
> +bool pci_dev_specific_ats_required(struct pci_dev *pdev)
> +{
> +	const struct pci_dev_ats_required *i;
> +
> +	for (i = pci_dev_ats_required; i->vendor; i++) {
> +		if (i->vendor != pdev->vendor)
> +			continue;
> +		if (i->ats_required && i->ats_required(pdev))
> +			return true;
> +		if (!i->ats_required && i->device == pdev->device)
> +			return true;
> +	}
> +
> +	return false;
> +}
>   #endif /* CONFIG_PCI_ATS */
>   
>   /* Freescale PCIe doesn't support MSI in RC mode */

[PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Nicolin Chen]

When a device's default substream attaches to an identity domain, the SMMU
driver currently sets the device's STE between two modes:

  Mode 1: Cfg=Translate, S1DSS=Bypass, EATS=1
  Mode 2: Cfg=bypass (EATS is ignored by HW)

When there is an active PASID (non-default substream), mode 1 is used. And
when there is no PASID support or no active PASID, mode 2 is used.

The driver will also downgrade an STE from mode 1 to mode 2, when the last
active substream becomes inactive.

However, there are PCIe devices that demand ATS to be always on. For these
devices, their STEs have to use the mode 1 as HW ignores EATS with mode 2.

Change the driver accordingly:
  - always use the mode 1
  - never downgrade to mode 2
  - allocate and retain a CD table (see note below)

Note that these devices might not support PASID, i.e. doing non-PASID ATS.
In such a case, the ssid_bits is set to 0. However, s1cdmax must be set to
a !0 value in order to keep the S1DSS field effective. Thus, when a master
requires ats_always_on, set its s1cdmax to at least 1, meaning that the CD
table will have a dummy entry (SSID=1) that will never be used.

Now for these devices, arm_smmu_cdtab_allocated() will always return true,
v.s. false prior to this change. When its default substream is attached to
an IDENTITY domain, its first CD is NULL in the table, which is a totally
valid case. Thus, add "!master->ats_always_on" to the condition.

Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
Acked-by: Nirmoy Das <nirmoyd@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  1 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 81 ++++++++++++++++++---
 2 files changed, 73 insertions(+), 9 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index ef42df4753ec4..8c3600f4364c5 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -943,6 +943,7 @@ struct arm_smmu_master {
 	bool				ats_enabled : 1;
 	bool				ste_ats_enabled : 1;
 	bool				stall_enabled;
+	bool				ats_always_on;
 	unsigned int			ssid_bits;
 	unsigned int			iopf_refcount;
 };
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index e8d7dbe495f03..4afdb775e0722 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1742,8 +1742,11 @@ void arm_smmu_clear_cd(struct arm_smmu_master *master, ioasid_t ssid)
 	if (!arm_smmu_cdtab_allocated(&master->cd_table))
 		return;
 	cdptr = arm_smmu_get_cd_ptr(master, ssid);
-	if (WARN_ON(!cdptr))
+	if (!cdptr) {
+		/* Only ats_always_on allows a NULL CD on default substream */
+		WARN_ON(!master->ats_always_on || ssid);
 		return;
+	}
 	arm_smmu_write_cd_entry(master, ssid, cdptr, &target);
 }
 
@@ -1756,6 +1759,22 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master)
 	struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table;
 
 	cd_table->s1cdmax = master->ssid_bits;
+
+	/*
+	 * When a device doesn't support PASID (non default SSID), ssid_bits is
+	 * set to 0. This also sets S1CDMAX to 0, which disables the substreams
+	 * and ignores the S1DSS field.
+	 *
+	 * On the other hand, if a device demands ATS to be always on even when
+	 * its default substream is IOMMU bypassed, it has to use EATS that is
+	 * only effective with an STE (CFG=S1translate, S1DSS=Bypass). For such
+	 * use cases, S1CDMAX has to be !0, in order to make use of S1DSS/EATS.
+	 *
+	 * Set S1CDMAX no lower than 1. This would add a dummy substream in the
+	 * CD table but it should never be used by an actual CD.
+	 */
+	if (master->ats_always_on)
+		cd_table->s1cdmax = max_t(u8, cd_table->s1cdmax, 1);
 	max_contexts = 1 << cd_table->s1cdmax;
 
 	if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) ||
@@ -3854,9 +3873,12 @@ static int arm_smmu_blocking_set_dev_pasid(struct iommu_domain *new_domain,
 	if (!arm_smmu_ssids_in_use(&master->cd_table)) {
 		struct iommu_domain *sid_domain =
 			iommu_driver_get_domain_for_dev(master->dev);
+		bool ats_always_on = master->ats_always_on &&
+				     sid_domain->type != IOMMU_DOMAIN_BLOCKED;
+		bool downgrade = sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
+				 sid_domain->type == IOMMU_DOMAIN_BLOCKED;
 
-		if (sid_domain->type == IOMMU_DOMAIN_IDENTITY ||
-		    sid_domain->type == IOMMU_DOMAIN_BLOCKED)
+		if (!ats_always_on && downgrade)
 			sid_domain->ops->attach_dev(sid_domain, dev,
 						    sid_domain);
 	}
@@ -3875,6 +3897,8 @@ static void arm_smmu_attach_dev_ste(struct iommu_domain *domain,
 		.old_domain = old_domain,
 		.ssid = IOMMU_NO_PASID,
 	};
+	bool ats_always_on = master->ats_always_on &&
+			     s1dss != STRTAB_STE_1_S1DSS_TERMINATE;
 
 	/*
 	 * Do not allow any ASID to be changed while are working on the STE,
@@ -3886,7 +3910,7 @@ static void arm_smmu_attach_dev_ste(struct iommu_domain *domain,
 	 * If the CD table is not in use we can use the provided STE, otherwise
 	 * we use a cdtable STE with the provided S1DSS.
 	 */
-	if (arm_smmu_ssids_in_use(&master->cd_table)) {
+	if (ats_always_on || arm_smmu_ssids_in_use(&master->cd_table)) {
 		/*
 		 * If a CD table has to be present then we need to run with ATS
 		 * on because we have to assume a PASID is using ATS. For
@@ -4215,6 +4239,44 @@ static void arm_smmu_remove_master(struct arm_smmu_master *master)
 	kfree(master->build_invs);
 }
 
+static int arm_smmu_master_prepare_ats(struct arm_smmu_master *master)
+{
+	bool s1p = master->smmu->features & ARM_SMMU_FEAT_TRANS_S1;
+	unsigned int stu = __ffs(master->smmu->pgsize_bitmap);
+	struct pci_dev *pdev;
+	int ret;
+
+	if (!dev_is_pci(master->dev))
+		return 0;
+	pdev = to_pci_dev(master->dev);
+
+	if (!arm_smmu_ats_supported(master)) {
+		if (pci_ats_required(pdev)) {
+			dev_err_once(master->dev, "SMMU doesn't support ATS\n");
+			return -EOPNOTSUPP;
+		}
+		return 0;
+	}
+
+	ret = pci_prepare_ats(pdev, stu);
+	if (ret || !pci_ats_required(pdev))
+		return ret;
+
+	/*
+	 * S1DSS is required for ATS to be always on for identity domain cases.
+	 * However, the S1DSS field is ignored if !IDR0_S1P or !IDR1_SSIDSIZE.
+	 */
+	if (!s1p || !master->smmu->ssid_bits) {
+		dev_err_once(master->dev,
+			     "SMMU doesn't support ATS to be always on\n");
+		return -EOPNOTSUPP;
+	}
+
+	master->ats_always_on = true;
+
+	return arm_smmu_alloc_cd_tables(master);
+}
+
 static struct iommu_device *arm_smmu_probe_device(struct device *dev)
 {
 	int ret;
@@ -4263,14 +4325,15 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
 	    smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
 		master->stall_enabled = true;
 
-	if (dev_is_pci(dev)) {
-		unsigned int stu = __ffs(smmu->pgsize_bitmap);
-
-		pci_prepare_ats(to_pci_dev(dev), stu);
-	}
+	ret = arm_smmu_master_prepare_ats(master);
+	if (ret)
+		goto err_disable_pasid;
 
 	return &smmu->iommu;
 
+err_disable_pasid:
+	arm_smmu_disable_pasid(master);
+	arm_smmu_remove_master(master);
 err_free_master:
 	kfree(master);
 	return ERR_PTR(ret);
-- 
2.43.0

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Pranjal Shrivastava]

On Thu, May 21, 2026 at 01:34:22PM -0700, Nicolin Chen wrote:
> When a device's default substream attaches to an identity domain, the SMMU
> driver currently sets the device's STE between two modes:
> 
>   Mode 1: Cfg=Translate, S1DSS=Bypass, EATS=1
>   Mode 2: Cfg=bypass (EATS is ignored by HW)
> 
> When there is an active PASID (non-default substream), mode 1 is used. And
> when there is no PASID support or no active PASID, mode 2 is used.
> 
> The driver will also downgrade an STE from mode 1 to mode 2, when the last
> active substream becomes inactive.
> 
> However, there are PCIe devices that demand ATS to be always on. For these
> devices, their STEs have to use the mode 1 as HW ignores EATS with mode 2.
> 
> Change the driver accordingly:
>   - always use the mode 1
>   - never downgrade to mode 2
>   - allocate and retain a CD table (see note below)
> 
> Note that these devices might not support PASID, i.e. doing non-PASID ATS.
> In such a case, the ssid_bits is set to 0. However, s1cdmax must be set to
> a !0 value in order to keep the S1DSS field effective. Thus, when a master
> requires ats_always_on, set its s1cdmax to at least 1, meaning that the CD
> table will have a dummy entry (SSID=1) that will never be used.
> 
> Now for these devices, arm_smmu_cdtab_allocated() will always return true,
> v.s. false prior to this change. When its default substream is attached to
> an IDENTITY domain, its first CD is NULL in the table, which is a totally
> valid case. Thus, add "!master->ats_always_on" to the condition.
> 
> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
> Tested-by: Nirmoy Das <nirmoyd@nvidia.com>
> Acked-by: Nirmoy Das <nirmoyd@nvidia.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ---
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  1 +
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 81 ++++++++++++++++++---
>  2 files changed, 73 insertions(+), 9 deletions(-)
>

Reviewed-by: Pranjal Shrivastava <praan@google.com>

>  
> +static int arm_smmu_master_prepare_ats(struct arm_smmu_master *master)
> +{
> +	bool s1p = master->smmu->features & ARM_SMMU_FEAT_TRANS_S1;
> +	unsigned int stu = __ffs(master->smmu->pgsize_bitmap);
> +	struct pci_dev *pdev;
> +	int ret;
> +
> +	if (!dev_is_pci(master->dev))
> +		return 0;
> +	pdev = to_pci_dev(master->dev);
> +
> +	if (!arm_smmu_ats_supported(master)) {
> +		if (pci_ats_required(pdev)) {
> +			dev_err_once(master->dev, "SMMU doesn't support ATS\n");
> +			return -EOPNOTSUPP;
> +		}
> +		return 0;
> +	}
> +
> +	ret = pci_prepare_ats(pdev, stu);
> +	if (ret || !pci_ats_required(pdev))
> +		return ret;
> +
> +	/*
> +	 * S1DSS is required for ATS to be always on for identity domain cases.
> +	 * However, the S1DSS field is ignored if !IDR0_S1P or !IDR1_SSIDSIZE.
> +	 */
> +	if (!s1p || !master->smmu->ssid_bits) {
> +		dev_err_once(master->dev,
> +			     "SMMU doesn't support ATS to be always on\n");
> +		return -EOPNOTSUPP;
> +	}
> +
> +	master->ats_always_on = true;
> +
> +	return arm_smmu_alloc_cd_tables(master);

Nit: I'm not sure if I'm getting this right, are we saying we *need* to
allocate CDs for CXL.cache cases in the probe itself because STE.EATS
requires Config=Translate with S1DSS in bypass?

Does this imply that active transactions can be occurring *before* the
first domain attach? Are 

Thanks,
Praan

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Jason Gunthorpe]

On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:

> > +	master->ats_always_on = true;
> > +
> > +	return arm_smmu_alloc_cd_tables(master);
> 
> Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> requires Config=Translate with S1DSS in bypass?

Yeah, basically.

Effectively the ARM rules for enabling ATS and Bypass together require
a valid CD table pointer.

Jason

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Pranjal Shrivastava]

On Thu, May 28, 2026 at 12:29:30PM -0300, Jason Gunthorpe wrote:
> On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:
> 
> > > +	master->ats_always_on = true;
> > > +
> > > +	return arm_smmu_alloc_cd_tables(master);
> > 
> > Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> > allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> > requires Config=Translate with S1DSS in bypass?
> 
> Yeah, basically.
> 
> Effectively the ARM rules for enabling ATS and Bypass together require
> a valid CD table pointer.

Ack. But do we expect transactions before attach_dev? What'd happen if
we don't allocate CDs here? Am I missing the fact that CXL hardware
could be already active and using ATS after the device_probe succeeds?

Thanks,
Praan

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Jason Gunthorpe]

On Thu, May 28, 2026 at 04:32:23PM +0000, Pranjal Shrivastava wrote:
> On Thu, May 28, 2026 at 12:29:30PM -0300, Jason Gunthorpe wrote:
> > On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:
> > 
> > > > +	master->ats_always_on = true;
> > > > +
> > > > +	return arm_smmu_alloc_cd_tables(master);
> > > 
> > > Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> > > allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> > > requires Config=Translate with S1DSS in bypass?
> > 
> > Yeah, basically.
> > 
> > Effectively the ARM rules for enabling ATS and Bypass together require
> > a valid CD table pointer.
> 
> Ack. But do we expect transactions before attach_dev? What'd happen if
> we don't allocate CDs here?

It can be done at any point before a non-abort STE is formed.

The core code pretty much does this immediately after probe so there
isn't much point in moving it later.

> Am I missing the fact that CXL hardware could be already active and
> using ATS after the device_probe succeeds?

The STEs are preloaded with abort when the driver starts, so that
should not be possible.

Jason

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Pranjal Shrivastava]

On Thu, May 28, 2026 at 03:00:18PM -0300, Jason Gunthorpe wrote:
> On Thu, May 28, 2026 at 04:32:23PM +0000, Pranjal Shrivastava wrote:
> > On Thu, May 28, 2026 at 12:29:30PM -0300, Jason Gunthorpe wrote:
> > > On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:
> > > 
> > > > > +	master->ats_always_on = true;
> > > > > +
> > > > > +	return arm_smmu_alloc_cd_tables(master);
> > > > 
> > > > Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> > > > allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> > > > requires Config=Translate with S1DSS in bypass?
> > > 
> > > Yeah, basically.
> > > 
> > > Effectively the ARM rules for enabling ATS and Bypass together require
> > > a valid CD table pointer.
> > 
> > Ack. But do we expect transactions before attach_dev? What'd happen if
> > we don't allocate CDs here?
> 
> It can be done at any point before a non-abort STE is formed.
> 
> The core code pretty much does this immediately after probe so there
> isn't much point in moving it later.
> 

Right.. that's what I'm not understanding clearly, if the STE is Abort
anyway during probe, why are we breaking the alloc-at-attach convention? 
Usually, we allocate CD tables in the first attach_dev call. Why does
that not work here?

[Disclaimer: Not a pushback, just trying to understand the requirement]

Praan

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Nicolin Chen]

On Thu, May 28, 2026 at 06:14:57PM +0000, Pranjal Shrivastava wrote:
> On Thu, May 28, 2026 at 03:00:18PM -0300, Jason Gunthorpe wrote:
> > On Thu, May 28, 2026 at 04:32:23PM +0000, Pranjal Shrivastava wrote:
> > > On Thu, May 28, 2026 at 12:29:30PM -0300, Jason Gunthorpe wrote:
> > > > On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:
> > > > 
> > > > > > +	master->ats_always_on = true;
> > > > > > +
> > > > > > +	return arm_smmu_alloc_cd_tables(master);
> > > > > 
> > > > > Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> > > > > allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> > > > > requires Config=Translate with S1DSS in bypass?
> > > > 
> > > > Yeah, basically.
> > > > 
> > > > Effectively the ARM rules for enabling ATS and Bypass together require
> > > > a valid CD table pointer.
> > > 
> > > Ack. But do we expect transactions before attach_dev? What'd happen if
> > > we don't allocate CDs here?
> > 
> > It can be done at any point before a non-abort STE is formed.
> > 
> > The core code pretty much does this immediately after probe so there
> > isn't much point in moving it later.
> > 
> 
> Right.. that's what I'm not understanding clearly, if the STE is Abort
> anyway during probe, why are we breaking the alloc-at-attach convention? 
> Usually, we allocate CD tables in the first attach_dev call. Why does
> that not work here?

It would work in attach_dev(). I just felt cleaner to have it in
probe(), paired with arm_smmu_free_cd_tables() in the release().

Thanks
Nicolin

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Jason Gunthorpe]

On Thu, May 28, 2026 at 11:20:35AM -0700, Nicolin Chen wrote:
> On Thu, May 28, 2026 at 06:14:57PM +0000, Pranjal Shrivastava wrote:
> > On Thu, May 28, 2026 at 03:00:18PM -0300, Jason Gunthorpe wrote:
> > > On Thu, May 28, 2026 at 04:32:23PM +0000, Pranjal Shrivastava wrote:
> > > > On Thu, May 28, 2026 at 12:29:30PM -0300, Jason Gunthorpe wrote:
> > > > > On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:
> > > > > 
> > > > > > > +	master->ats_always_on = true;
> > > > > > > +
> > > > > > > +	return arm_smmu_alloc_cd_tables(master);
> > > > > > 
> > > > > > Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> > > > > > allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> > > > > > requires Config=Translate with S1DSS in bypass?
> > > > > 
> > > > > Yeah, basically.
> > > > > 
> > > > > Effectively the ARM rules for enabling ATS and Bypass together require
> > > > > a valid CD table pointer.
> > > > 
> > > > Ack. But do we expect transactions before attach_dev? What'd happen if
> > > > we don't allocate CDs here?
> > > 
> > > It can be done at any point before a non-abort STE is formed.
> > > 
> > > The core code pretty much does this immediately after probe so there
> > > isn't much point in moving it later.
> > > 
> > 
> > Right.. that's what I'm not understanding clearly, if the STE is Abort
> > anyway during probe, why are we breaking the alloc-at-attach convention? 
> > Usually, we allocate CD tables in the first attach_dev call. Why does
> > that not work here?
> 
> It would work in attach_dev(). I just felt cleaner to have it in
> probe(), paired with arm_smmu_free_cd_tables() in the release().

I thought it was like this because we don't have an alloc on attach
flow for the identity domain, so add it there or add it here..

Jason

Re: [PATCH v6 3/3] iommu/arm-smmu-v3: Allow ATS to be always on

[Pranjal Shrivastava]

On Thu, May 28, 2026 at 05:04:50PM -0300, Jason Gunthorpe wrote:
> On Thu, May 28, 2026 at 11:20:35AM -0700, Nicolin Chen wrote:
> > On Thu, May 28, 2026 at 06:14:57PM +0000, Pranjal Shrivastava wrote:
> > > On Thu, May 28, 2026 at 03:00:18PM -0300, Jason Gunthorpe wrote:
> > > > On Thu, May 28, 2026 at 04:32:23PM +0000, Pranjal Shrivastava wrote:
> > > > > On Thu, May 28, 2026 at 12:29:30PM -0300, Jason Gunthorpe wrote:
> > > > > > On Thu, May 28, 2026 at 03:24:40PM +0000, Pranjal Shrivastava wrote:
> > > > > > 
> > > > > > > > +	master->ats_always_on = true;
> > > > > > > > +
> > > > > > > > +	return arm_smmu_alloc_cd_tables(master);
> > > > > > > 
> > > > > > > Nit: I'm not sure if I'm getting this right, are we saying we *need* to
> > > > > > > allocate CDs for CXL.cache cases in the probe itself because STE.EATS
> > > > > > > requires Config=Translate with S1DSS in bypass?
> > > > > > 
> > > > > > Yeah, basically.
> > > > > > 
> > > > > > Effectively the ARM rules for enabling ATS and Bypass together require
> > > > > > a valid CD table pointer.
> > > > > 
> > > > > Ack. But do we expect transactions before attach_dev? What'd happen if
> > > > > we don't allocate CDs here?
> > > > 
> > > > It can be done at any point before a non-abort STE is formed.
> > > > 
> > > > The core code pretty much does this immediately after probe so there
> > > > isn't much point in moving it later.
> > > > 
> > > 
> > > Right.. that's what I'm not understanding clearly, if the STE is Abort
> > > anyway during probe, why are we breaking the alloc-at-attach convention? 
> > > Usually, we allocate CD tables in the first attach_dev call. Why does
> > > that not work here?
> > 
> > It would work in attach_dev(). I just felt cleaner to have it in
> > probe(), paired with arm_smmu_free_cd_tables() in the release().
> 
> I thought it was like this because we don't have an alloc on attach
> flow for the identity domain, so add it there or add it here..

Right, doing it in attach would've been cleaner, but I don't mind the
existing way too (since it's merged anyway)

I went down this hole since the reason for the convention change wasn't
immediately clear.

Thanks for the clarification!
Praan

Re: [PATCH v6 0/3] Allow ATS to be always on for certain ATS-capable devices

[Jörg Rödel]

On Thu, May 21, 2026 at 01:34:19PM -0700, Nicolin Chen wrote:
> Nicolin Chen (3):
>   PCI: Add pci_ats_required() for CXL.cache capable devices
>   PCI: Allow ATS to be always on for pre-CXL devices
>   iommu/arm-smmu-v3: Allow ATS to be always on
> 
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  1 +
>  drivers/pci/pci.h                           |  9 +++
>  include/linux/pci-ats.h                     |  3 +
>  include/uapi/linux/pci_regs.h               |  1 +
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 81 ++++++++++++++++++---
>  drivers/pci/ats.c                           | 47 ++++++++++++
>  drivers/pci/quirks.c                        | 42 +++++++++++
>  7 files changed, 175 insertions(+), 9 deletions(-)

Applied, thanks.