[PATCH v20 00/16] vfio/pci: Add PCIe TPH support

[PATCH v20 00/16] vfio/pci: Add PCIe TPH support

[Chengwen Feng]

This patchset enables full userspace configurable PCIe TPH support for
VFIO, which brings performance benefits for userspace workloads such as
DPDK and SPDK.

Currently VFIO only exposes read-only TPH capability registers to
userspace, while all write operations are silently discarded. This
prevents userspace from enabling and configuring TPH, limiting performance
optimization opportunities.

Per PCIe spec 7.5.3.15: TPH Completer support is applicable to Root Ports
and Endpoints, allowing Steering Tags to target host CPUs or peer devices
for P2P transactions.

TPH usage model can be divided into three fundamental parts:
1. Retrieve Steering Tag:
   - Tags targeting host CPUs are obtained via platform methods (ACPI _DSM)
     wrapped in pcie_tph_get_cpu_st(). Userspace requires a generic
     interface to query these CPU-associated ST values.
   - Tags targeting peer devices are managed by userspace drivers, and set
     to kernel by dmabuf ioctl which address by zhiping's patchset.
2. Program Steering Tag table:
   - For devices with standard ST table structures (in capability space or
     MSI-X table), userspace needs a unified interface to configure ST
     entries.
   - Devices without standard ST tables are handled by userspace itself.
3. Toggle device TPH Requester enable/disable state.

The patchset builds a complete, secure, staged TPH feature stack with core
design objectives:
1. Refactor & fix upstream PCI/TPH core implementation to provide clean,
   safe, extendable helper APIs for VFIO consumption;
2. Introduce configurable per-device TPH policy for staged feature
   enablement, supporting progressive enabling of No-ST/IV/DS/LITERAL
   modes;
3. Fully virtualize PCIe TPH config space: read-only capability header,
   virtually writable TPH_CTRL, silent discard invalid writes;
4. Implement unified VFIO user feature uAPI (TPH / TPH_RESOLVE / TPH_ST)
   for TPH enablement, tag resolution and batch ST table programming;

The patchset is split into three logical parts:
1. The first three patches is Zhiping's patchset of add TPH support for
   p2p access (based on v11), because this patchset support query dmabuf's
   ST/PH
2. The second eight patches fix and refactor core PCI/TPH kernel code to
   export required helper interfaces
3. The last five patches implement corresponding VFIO TPH virtualization
   and uAPI step by step.

Based on earlier RFC work by Wathsala Vithanage

---
v20:
- Address almost all comments from Alex about v19, not within Alex's
  scope:
  - Drop LITERAL policy support
  - uAPI changes: cap have been split into fine-grained
  - TPH config access: still based on perm mechanism, but restrict the
    readable and writable length
  - Debugfs add more fields
v19:
- Address all comments from Alex
  - Drop CPU-to-ST translation of PCI/TPH
  - Re-implement vfio part, please refer:
    https://lore.kernel.org/linux-pci/20260626092232.53ed3a7c@shazbot.org/
- Add expose tph_policy via debugfs

Chengwen Feng (13):
  PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction
  PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing
  PCI/TPH: Cache negotiated max requester type at probe time
  PCI/TPH: Refactor pcie_enable_tph and add requester variant
  PCI/TPH: Refactor pcie_tph_get_cpu_st & add requester variant
  PCI/TPH: Add macros to check standard and extended TPH support
  PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming
    helper
  PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST
    _DSM
  vfio/pci: Add basic NO-ST TPH VFIO device feature support
  vfio/pci: Add IV-ST TPH policy and VFIO TPH_ST batch programming
    support
  vfio/pci: Add DS-ST TPH policy and extend TPH_RESOLVE capabilities
  vfio/pci: Sync core PCI TPH software state across device lifecycle and
    resets
  vfio/pci: Expose per-device TPH state via debugfs

Zhiping Zhang (3):
  PCI/TPH: Add requester/completer type helpers
  dma-buf: add optional get_pci_tph() callback
  vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature

 drivers/dma-buf/dma-buf.c          |  32 +++
 drivers/pci/tph.c                  | 282 ++++++++++++++++++------
 drivers/vfio/pci/vfio_pci.c        |  23 ++
 drivers/vfio/pci/vfio_pci_config.c | 164 ++++++++++++--
 drivers/vfio/pci/vfio_pci_core.c   | 341 ++++++++++++++++++++++++++++-
 drivers/vfio/pci/vfio_pci_dmabuf.c | 102 ++++++++-
 drivers/vfio/pci/vfio_pci_priv.h   |  18 ++
 include/linux/dma-buf.h            |  25 +++
 include/linux/pci-tph.h            |  53 ++++-
 include/linux/pci.h                |   6 +-
 include/linux/vfio_pci_core.h      |   3 +
 include/uapi/linux/vfio.h          | 144 ++++++++++++
 12 files changed, 1090 insertions(+), 103 deletions(-)

-- 
2.17.1

[PATCH v20 01/16] PCI/TPH: Add requester/completer type helpers

[Chengwen Feng]

From: Zhiping Zhang <zhipingz@meta.com>

Add pcie_tph_enabled_req_type() so drivers can query the enabled TPH
requester mode without reaching into pci_dev internals.

Add pcie_tph_completer_type() so drivers that publish TPH metadata for
a device acting as a completer can gate on the "TPH Completer
Supported" field of Device Capabilities 2 (bits 13:12,
PCI_EXP_DEVCAP2_TPH_COMP_MASK) rather than reusing requester-side
state. Fold the reserved 0b10 encoding into NONE so callers only see
the defined values.

This keeps pci_dev::tph_req_type and the completer-capability decode
inside the PCI/TPH code and provides !CONFIG_PCIE_TPH stubs for
callers.

Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
---
 drivers/pci/tph.c       | 45 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci-tph.h |  8 ++++++++
 2 files changed, 53 insertions(+)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 655ffd60e62f..e7693fd9d676 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -173,6 +173,51 @@ u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
 }
 EXPORT_SYMBOL(pcie_tph_get_st_table_loc);
 
+/**
+ * pcie_tph_enabled_req_type - Return the device's enabled TPH requester type
+ * @pdev: PCI device to query
+ *
+ * Return: PCI_TPH_REQ_DISABLE, PCI_TPH_REQ_TPH_ONLY or PCI_TPH_REQ_EXT_TPH.
+ */
+u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
+{
+	return pdev->tph_req_type;
+}
+EXPORT_SYMBOL(pcie_tph_enabled_req_type);
+
+/**
+ * pcie_tph_completer_type - Return the device's TPH Completer support
+ * @pdev: PCI device to query
+ *
+ * Reads the "TPH Completer Supported" field (bits 13:12) of Device
+ * Capabilities 2. The reserved 0b10 encoding is folded into
+ * "not supported" so callers only need to compare against the three
+ * defined values.
+ *
+ * Return: one of %PCI_EXP_DEVCAP2_TPH_COMP_NONE,
+ *         %PCI_EXP_DEVCAP2_TPH_COMP_TPH_ONLY or
+ *         %PCI_EXP_DEVCAP2_TPH_COMP_EXT_TPH.
+ */
+u8 pcie_tph_completer_type(struct pci_dev *pdev)
+{
+	u32 reg;
+
+	if (pcie_capability_read_dword(pdev, PCI_EXP_DEVCAP2, &reg))
+		return PCI_EXP_DEVCAP2_TPH_COMP_NONE;
+	if (PCI_POSSIBLE_ERROR(reg))
+		return PCI_EXP_DEVCAP2_TPH_COMP_NONE;
+
+	switch (FIELD_GET(PCI_EXP_DEVCAP2_TPH_COMP_MASK, reg)) {
+	case PCI_EXP_DEVCAP2_TPH_COMP_TPH_ONLY:
+		return PCI_EXP_DEVCAP2_TPH_COMP_TPH_ONLY;
+	case PCI_EXP_DEVCAP2_TPH_COMP_EXT_TPH:
+		return PCI_EXP_DEVCAP2_TPH_COMP_EXT_TPH;
+	default:
+		return PCI_EXP_DEVCAP2_TPH_COMP_NONE;
+	}
+}
+EXPORT_SYMBOL(pcie_tph_completer_type);
+
 /*
  * Return the size of ST table. If ST table is not in TPH Requester Extended
  * Capability space, return 0. Otherwise return the ST Table Size + 1.
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index be68cd17f2f8..7743af6fe432 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -9,6 +9,8 @@
 #ifndef LINUX_PCI_TPH_H
 #define LINUX_PCI_TPH_H
 
+#include <linux/pci_regs.h>
+
 /*
  * According to the ECN for PCI Firmware Spec, Steering Tag can be different
  * depending on the memory type: Volatile Memory or Persistent Memory. When a
@@ -30,6 +32,8 @@ void pcie_disable_tph(struct pci_dev *pdev);
 int pcie_enable_tph(struct pci_dev *pdev, int mode);
 u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
+u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
+u8 pcie_tph_completer_type(struct pci_dev *pdev);
 #else
 static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
 					unsigned int index, u16 tag)
@@ -41,6 +45,10 @@ static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
 static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
 { return -EINVAL; }
+static inline u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
+{ return PCI_TPH_REQ_DISABLE; }
+static inline u8 pcie_tph_completer_type(struct pci_dev *pdev)
+{ return PCI_EXP_DEVCAP2_TPH_COMP_NONE; }
 #endif
 
 #endif /* LINUX_PCI_TPH_H */
-- 
2.17.1

[PATCH v20 02/16] dma-buf: add optional get_pci_tph() callback

[Chengwen Feng]

From: Zhiping Zhang <zhipingz@meta.com>

Add an optional dma_buf_ops.get_pci_tph callback and a
DMA-buf importer wrapper, dma_buf_get_pci_tph().

TPH is PCIe TLP Processing Hint. 8-bit ST and 16-bit Extended ST are
distinct PCIe TPH namespaces, so the importer requests the namespace it
can emit and the exporter returns the matching ST/PH tuple or
-EOPNOTSUPP.

dma_buf_get_pci_tph() is the importer entry point. It requires
&dmabuf->resv to be held while the callback runs and returns
-EOPNOTSUPP when the exporter does not provide PCI TPH metadata.

The first user is VFIO_DEVICE_FEATURE_DMA_BUF_TPH in vfio-pci, with
mlx5 as the first importer.

Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
---
 drivers/dma-buf/dma-buf.c | 32 ++++++++++++++++++++++++++++++++
 include/linux/dma-buf.h   | 25 +++++++++++++++++++++++++
 2 files changed, 57 insertions(+)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index d504c636dc29..e8314ed240d3 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -939,6 +939,7 @@ dma_buf_pin_on_map(struct dma_buf_attachment *attach)
  *     - dma_buf_unmap_attachment()
  *     - dma_buf_vmap()
  *     - dma_buf_vunmap()
+ *     - dma_buf_get_pci_tph()
  *
  * 2. Importers must not hold the dma-buf reservation lock when calling these
  *    functions:
@@ -979,6 +980,7 @@ dma_buf_pin_on_map(struct dma_buf_attachment *attach)
  *     - &dma_buf_ops.unmap_dma_buf()
  *     - &dma_buf_ops.vmap()
  *     - &dma_buf_ops.vunmap()
+ *     - &dma_buf_ops.get_pci_tph()
  *
  * 3. Exporters must hold the dma-buf reservation lock when calling these
  *    functions:
@@ -1144,6 +1146,36 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, "DMA_BUF");
 
+/**
+ * dma_buf_get_pci_tph - Retrieve PCIe TLP Processing Hint (TPH) metadata
+ * @dmabuf: DMA buffer to query
+ * @extended: false for 8-bit ST, true for 16-bit Extended ST
+ * @steering_tag: returns the raw steering tag for the requested namespace
+ * @ph: returns the TPH processing hint
+ *
+ * Wrapper for the optional &dma_buf_ops.get_pci_tph callback.
+ *
+ * Must be called with &dma_buf.resv held. Returns -EOPNOTSUPP if the
+ * exporter does not implement the callback or has no metadata for the
+ * requested namespace.
+ *
+ * The returned steering tag and processing hint are only valid until the
+ * exporter invalidates the current mapping (signalled to the importer via
+ * &dma_buf_attach_ops.move_notify). Importers must re-query after a new
+ * mapping is established following invalidation.
+ */
+int dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
+			u16 *steering_tag, u8 *ph)
+{
+	dma_resv_assert_held(dmabuf->resv);
+
+	if (!dmabuf->ops->get_pci_tph)
+		return -EOPNOTSUPP;
+
+	return dmabuf->ops->get_pci_tph(dmabuf, extended, steering_tag, ph);
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_get_pci_tph, "DMA_BUF");
+
 /**
  * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
  * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index d1203da56fc5..14e8eb61a62c 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -113,6 +113,29 @@ struct dma_buf_ops {
 	 */
 	void (*unpin)(struct dma_buf_attachment *attach);
 
+	/**
+	 * @get_pci_tph:
+	 *
+	 * Retrieve PCIe TLP Processing Hint (TPH) steering metadata for
+	 * this buffer so an importer can program a matching ST/PH hint on
+	 * outbound TLPs targeting the exporter for peer-to-peer DMA.
+	 *
+	 * @dmabuf: DMA buffer for which to retrieve TPH metadata
+	 * @extended: false for 8-bit ST, true for 16-bit Extended ST
+	 * @steering_tag: Returns the raw TPH steering tag for the requested
+	 *                namespace
+	 * @ph: Returns the TPH processing hint (2-bit value)
+	 *
+	 * Optional callback for dma_buf_get_pci_tph(). Called with
+	 * &dma_buf.resv held. Returns 0 on success or -EOPNOTSUPP when
+	 * the exporter has no metadata for the requested namespace.
+	 *
+	 * Returned metadata follows the lifetime rules documented for
+	 * dma_buf_get_pci_tph().
+	 */
+	int (*get_pci_tph)(struct dma_buf *dmabuf, bool extended,
+			   u16 *steering_tag, u8 *ph);
+
 	/**
 	 * @map_dma_buf:
 	 *
@@ -563,6 +586,8 @@ void dma_buf_detach(struct dma_buf *dmabuf,
 		    struct dma_buf_attachment *attach);
 int dma_buf_pin(struct dma_buf_attachment *attach);
 void dma_buf_unpin(struct dma_buf_attachment *attach);
+int dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
+			u16 *steering_tag, u8 *ph);
 
 struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info);
 
-- 
2.17.1

[PATCH v20 03/16] vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature

[Chengwen Feng]

From: Zhiping Zhang <zhipingz@meta.com>

Implement dma-buf get_pci_tph for vfio-pci exported dma-bufs and add
VFIO_DEVICE_FEATURE_DMA_BUF_TPH so userspace can publish TPH metadata
for a VFIO-owned device.

8-bit ST and 16-bit Extended ST are distinct PCIe TPH namespaces; the
uAPI carries both with explicit validity flags, and get_pci_tph()
returns the value matching the importer's requested namespace or
-EOPNOTSUPP.

Publish and read the TPH descriptor under dmabuf->resv, matching the
locking used for other importer-visible dma-buf state. The SET ioctl
takes dma_resv_lock_interruptible(), while the callback runs under
DMA-buf's asserted resv lock.

The attach path reads @revoked without holding memory_lock. Annotate it
with READ_ONCE() to document this intentional lockless access: the read
is a benign early-out, and a racing revocation is re-checked under
dmabuf->resv in vfio_pci_dma_buf_map() before any mapping is handed out.
The annotation only needs to keep the access well-formed against the
memory_lock-protected writers.

Reject requests the device cannot consume as a completer:
pcie_tph_completer_type() must report at least
PCI_EXP_DEVCAP2_TPH_COMP_TPH_ONLY, and Extended ST requires
PCI_EXP_DEVCAP2_TPH_COMP_EXT_TPH. Make PROBE follow the same hardware
gate so the feature only probes as supported when the device can really
consume it.

Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
---
 drivers/vfio/pci/vfio_pci_core.c   |   3 +
 drivers/vfio/pci/vfio_pci_dmabuf.c | 102 ++++++++++++++++++++++++++++-
 drivers/vfio/pci/vfio_pci_priv.h   |  12 ++++
 include/uapi/linux/vfio.h          |  43 ++++++++++++
 4 files changed, 158 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 3f11a9624b9c..4e5e34a77b76 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1625,6 +1625,9 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 		return vfio_pci_core_feature_token(vdev, flags, arg, argsz);
 	case VFIO_DEVICE_FEATURE_DMA_BUF:
 		return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz);
+	case VFIO_DEVICE_FEATURE_DMA_BUF_TPH:
+		return vfio_pci_core_feature_dma_buf_tph(vdev, flags, arg,
+							 argsz);
 	default:
 		return -ENOTTY;
 	}
diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
index c16f460c01d6..5351839c1338 100644
--- a/drivers/vfio/pci/vfio_pci_dmabuf.c
+++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
@@ -3,6 +3,7 @@
  */
 #include <linux/dma-buf-mapping.h>
 #include <linux/pci-p2pdma.h>
+#include <linux/pci-tph.h>
 #include <linux/dma-resv.h>
 
 #include "vfio_pci_priv.h"
@@ -19,7 +20,17 @@ struct vfio_pci_dma_buf {
 	u32 nr_ranges;
 	struct kref kref;
 	struct completion comp;
-	u8 revoked : 1;
+
+	/*
+	 * Updates protected by dmabuf->resv, @revoked additionally
+	 * protected by memory_lock.
+	 */
+	u16 tph_st_ext;
+	u8 tph_st;
+	bool revoked;
+	u8 tph_st_valid:1;
+	u8 tph_st_ext_valid:1;
+	u8 tph_ph:2;
 };
 
 static int vfio_pci_dma_buf_attach(struct dma_buf *dmabuf,
@@ -30,7 +41,7 @@ static int vfio_pci_dma_buf_attach(struct dma_buf *dmabuf,
 	if (!attachment->peer2peer)
 		return -EOPNOTSUPP;
 
-	if (priv->revoked)
+	if (READ_ONCE(priv->revoked))
 		return -ENODEV;
 
 	if (!dma_buf_attach_revocable(attachment))
@@ -69,6 +80,26 @@ vfio_pci_dma_buf_map(struct dma_buf_attachment *attachment,
 	return ret;
 }
 
+static int vfio_pci_dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
+					u16 *steering_tag, u8 *ph)
+{
+	struct vfio_pci_dma_buf *priv = dmabuf->priv;
+
+	dma_resv_assert_held(dmabuf->resv);
+
+	if (extended) {
+		if (!priv->tph_st_ext_valid)
+			return -EOPNOTSUPP;
+		*steering_tag = priv->tph_st_ext;
+	} else {
+		if (!priv->tph_st_valid)
+			return -EOPNOTSUPP;
+		*steering_tag = priv->tph_st;
+	}
+	*ph = priv->tph_ph;
+	return 0;
+}
+
 static void vfio_pci_dma_buf_unmap(struct dma_buf_attachment *attachment,
 				   struct sg_table *sgt,
 				   enum dma_data_direction dir)
@@ -101,6 +132,7 @@ static void vfio_pci_dma_buf_release(struct dma_buf *dmabuf)
 
 static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
 	.attach = vfio_pci_dma_buf_attach,
+	.get_pci_tph = vfio_pci_dma_buf_get_pci_tph,
 	.map_dma_buf = vfio_pci_dma_buf_map,
 	.unmap_dma_buf = vfio_pci_dma_buf_unmap,
 	.release = vfio_pci_dma_buf_release,
@@ -333,6 +365,72 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 	return ret;
 }
 
+int vfio_pci_core_feature_dma_buf_tph(struct vfio_pci_core_device *vdev,
+				      u32 flags,
+				      struct vfio_device_feature_dma_buf_tph __user *arg,
+				      size_t argsz)
+{
+	struct vfio_device_feature_dma_buf_tph set_tph;
+	struct vfio_pci_dma_buf *priv;
+	struct dma_buf *dmabuf;
+	u8 comp;
+	int ret;
+
+	comp = pcie_tph_completer_type(vdev->pdev);
+	if (comp == PCI_EXP_DEVCAP2_TPH_COMP_NONE)
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET,
+				 sizeof(set_tph));
+	if (ret != 1)
+		return ret;
+
+	if (copy_from_user(&set_tph, arg, sizeof(set_tph)))
+		return -EFAULT;
+
+	if (set_tph.flags & ~(VFIO_DMA_BUF_TPH_ST | VFIO_DMA_BUF_TPH_ST_EXT))
+		return -EINVAL;
+
+	if (set_tph.ph & ~0x3)
+		return -EINVAL;
+
+	if ((set_tph.flags & VFIO_DMA_BUF_TPH_ST_EXT) &&
+	    comp != PCI_EXP_DEVCAP2_TPH_COMP_EXT_TPH)
+		return -EOPNOTSUPP;
+
+	dmabuf = dma_buf_get(set_tph.dmabuf_fd);
+	if (IS_ERR(dmabuf))
+		return PTR_ERR(dmabuf);
+
+	if (dmabuf->ops != &vfio_pci_dmabuf_ops) {
+		ret = -EINVAL;
+		goto out_put;
+	}
+
+	priv = dmabuf->priv;
+	if (priv->vdev != vdev) {
+		ret = -EINVAL;
+		goto out_put;
+	}
+
+	ret = dma_resv_lock_interruptible(dmabuf->resv, NULL);
+	if (ret)
+		goto out_put;
+
+	priv->tph_st         = set_tph.steering_tag;
+	priv->tph_st_ext     = set_tph.steering_tag_ext;
+	priv->tph_ph         = set_tph.ph;
+	priv->tph_st_valid   = !!(set_tph.flags & VFIO_DMA_BUF_TPH_ST);
+	priv->tph_st_ext_valid =
+		!!(set_tph.flags & VFIO_DMA_BUF_TPH_ST_EXT);
+	dma_resv_unlock(dmabuf->resv);
+	ret = 0;
+
+out_put:
+	dma_buf_put(dmabuf);
+	return ret;
+}
+
 void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked)
 {
 	struct vfio_pci_dma_buf *priv;
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index fca9d0dfac90..c58f369be4b3 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -118,6 +118,10 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pdev)
 int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 				  struct vfio_device_feature_dma_buf __user *arg,
 				  size_t argsz);
+int vfio_pci_core_feature_dma_buf_tph(struct vfio_pci_core_device *vdev,
+				      u32 flags,
+				      struct vfio_device_feature_dma_buf_tph __user *arg,
+				      size_t argsz);
 void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev);
 void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked);
 #else
@@ -128,6 +132,14 @@ vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 {
 	return -ENOTTY;
 }
+
+static inline int
+vfio_pci_core_feature_dma_buf_tph(struct vfio_pci_core_device *vdev, u32 flags,
+				  struct vfio_device_feature_dma_buf_tph __user *arg,
+				  size_t argsz)
+{
+	return -ENOTTY;
+}
 static inline void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev)
 {
 }
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 5de618a3a5ee..4c1c70aac150 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -1534,6 +1534,49 @@ struct vfio_device_feature_dma_buf {
  */
 #define VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2  12
 
+/**
+ * Upon VFIO_DEVICE_FEATURE_SET associate TPH (TLP Processing Hints) metadata
+ * with a vfio-exported dma-buf. The dma-buf must have been created by
+ * VFIO_DEVICE_FEATURE_DMA_BUF on this device, and the device must report
+ * TPH Completer support in Device Capabilities 2 (bits 13:12); requests
+ * carrying VFIO_DMA_BUF_TPH_ST_EXT additionally require the device to
+ * report the Extended TPH Completer encoding. Otherwise the ioctl
+ * returns -EOPNOTSUPP.
+ *
+ * @dmabuf_fd is the file descriptor returned by VFIO_DEVICE_FEATURE_DMA_BUF.
+ *
+ * 8-bit ST (@steering_tag) and 16-bit Extended ST (@steering_tag_ext) are
+ * distinct namespaces. @flags is the authoritative validity mask for the
+ * two namespaces: VFIO_DMA_BUF_TPH_ST marks @steering_tag valid and
+ * VFIO_DMA_BUF_TPH_ST_EXT marks @steering_tag_ext valid. An importer
+ * requests one namespace and receives the matching value.
+ *
+ * Each SET fully replaces the dma-buf's TPH state for future queries
+ * only. A namespace whose flag bit is clear is reported as unsupported
+ * to future importers; an importer that has already retrieved a value
+ * is unaffected. @flags == 0 leaves both namespaces unsupported for
+ * future importers.
+ *
+ * @ph is the 2-bit TLP Processing Hint and must be in the range [0, 3].
+ *
+ * Undefined @flags and @ph bits must always be zero.
+ *
+ * Userspace must configure TPH before handing the dma-buf fd to an
+ * importer.
+ */
+#define VFIO_DEVICE_FEATURE_DMA_BUF_TPH 13
+
+#define VFIO_DMA_BUF_TPH_ST		(1 << 0)
+#define VFIO_DMA_BUF_TPH_ST_EXT		(1 << 1)
+
+struct vfio_device_feature_dma_buf_tph {
+	__s32	dmabuf_fd;
+	__u32	flags;
+	__u16	steering_tag_ext;
+	__u8	steering_tag;
+	__u8	ph;
+};
+
 /* -------- API for Type1 VFIO IOMMU -------- */
 
 /**
-- 
2.17.1

[PATCH v20 04/16] PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction

[Chengwen Feng]

pcie_tph_get_st_table_loc() incorrectly uses FIELD_GET(), which shifts the
field value to bit 0. But the function is designed to return raw
PCI_TPH_LOC_* values as defined in the function comment.

This causes incorrect ST table location detection. Fix it by using bitwise
AND with PCI_TPH_CAP_LOC_MASK to return the unshifted field value matching
the function specification.

This doesn't make a difference to mlx5_st_create(), the lone external
caller, because it only checks for PCI_TPH_LOC_NONE (0), but will be needed
for callers that check for PCI_TPH_LOC_CAP or PCI_TPH_LOC_MSIX.

Also add tph_cap validation for pcie_tph_get_st_table_loc() to prevent
invalid PCI configuration space access when TPH is not supported. Add stub
functions for pcie_tph_get_st_table_size() and pcie_tph_get_st_table_loc()
when !CONFIG_PCIE_TPH.

Fixes: d2e8a34876ce ("PCI/TPH: Add Steering Tag support")
Cc: stable@vger.kernel.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Reviewed-by: Alex Williamson <alex.williamson@nvidia.com>
Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/tph.c       | 12 +++++-------
 include/linux/pci-tph.h |  4 ++++
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index e7693fd9d676..41084a45b0bb 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -165,11 +165,14 @@ static u8 get_st_modes(struct pci_dev *pdev)
  */
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
 {
-	u32 reg;
+	u32 reg = 0;
+
+	if (!pdev->tph_cap)
+		return PCI_TPH_LOC_NONE;
 
 	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
 
-	return FIELD_GET(PCI_TPH_CAP_LOC_MASK, reg);
+	return reg & PCI_TPH_CAP_LOC_MASK;
 }
 EXPORT_SYMBOL(pcie_tph_get_st_table_loc);
 
@@ -229,9 +232,6 @@ u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
 
 	/* Check ST table location first */
 	loc = pcie_tph_get_st_table_loc(pdev);
-
-	/* Convert loc to match with PCI_TPH_LOC_* defined in pci_regs.h */
-	loc = FIELD_PREP(PCI_TPH_CAP_LOC_MASK, loc);
 	if (loc != PCI_TPH_LOC_CAP)
 		return 0;
 
@@ -360,8 +360,6 @@ int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
 	set_ctrl_reg_req_en(pdev, PCI_TPH_REQ_DISABLE);
 
 	loc = pcie_tph_get_st_table_loc(pdev);
-	/* Convert loc to match with PCI_TPH_LOC_* */
-	loc = FIELD_PREP(PCI_TPH_CAP_LOC_MASK, loc);
 
 	switch (loc) {
 	case PCI_TPH_LOC_MSIX:
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 7743af6fe432..0e6aa1895050 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -45,6 +45,10 @@ static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
 static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
 { return -EINVAL; }
+static inline u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
+{ return 0; }
+static inline u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
+{ return PCI_TPH_LOC_NONE; }
 static inline u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
 { return PCI_TPH_REQ_DISABLE; }
 static inline u8 pcie_tph_completer_type(struct pci_dev *pdev)
-- 
2.17.1

[PATCH v20 05/16] PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing

[Chengwen Feng]

Split tph_enabled from shared pci_dev bitfield into spare bit of tph_cap's
u16: tph_cap is immutable post-enumeration (15 bits for offset), remaining
1 bit stores tph_enabled. Removes cross-bitfield concurrent write hazards
highlighted by Sashiko after VFIO TPH exposure. No functional changes.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 include/linux/pci.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/pci.h b/include/linux/pci.h
index 64b308b6e61c..ce520523b152 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -472,7 +472,6 @@ struct pci_dev {
 	unsigned int	ats_enabled:1;		/* Address Translation Svc */
 	unsigned int	pasid_enabled:1;	/* Process Address Space ID */
 	unsigned int	pri_enabled:1;		/* Page Request Interface */
-	unsigned int	tph_enabled:1;		/* TLP Processing Hints */
 	unsigned int	fm_enabled:1;		/* Flit Mode (segment captured) */
 	unsigned int	is_managed:1;		/* Managed via devres */
 	unsigned int	is_msi_managed:1;	/* MSI release via devres installed */
@@ -588,7 +587,8 @@ struct pci_dev {
 	u8 reset_methods[PCI_NUM_RESET_METHODS]; /* In priority order */
 
 #ifdef CONFIG_PCIE_TPH
-	u16		tph_cap;	/* TPH capability offset */
+	u16		tph_cap:15;	/* TPH capability offset */
+	u16		tph_enabled:1;	/* Whether TPH is enabled */
 	u8		tph_mode;	/* TPH mode */
 	u8		tph_req_type;	/* TPH requester type */
 #endif
-- 
2.17.1

[PATCH v20 06/16] PCI/TPH: Cache negotiated max requester type at probe time

[Chengwen Feng]

pcie_enable_tph() recalculates the TPH requester capability and queries
root port completer type on every invocation, causing redundant config
space reads.

Calculate the negotiated maximum requester type once during pci_tph_init()
and cache it in a new pdev->tph_max_type bitfield. Simplify the enable path
by reusing the cached value to eliminate repeated hardware access.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c   | 45 ++++++++++++++++++++++++---------------------
 include/linux/pci.h |  4 +++-
 2 files changed, 27 insertions(+), 22 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 41084a45b0bb..d123f91775c6 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -418,9 +418,8 @@ EXPORT_SYMBOL(pcie_disable_tph);
  *   - PCI_TPH_ST_DS_MODE: Device Specific Mode
  *
  * Check whether the mode is actually supported by the device before enabling
- * and return an error if not. Additionally determine what types of requests,
- * TPH or extended TPH, can be issued by the device based on its TPH requester
- * capability and the Root Port's completer capability.
+ * and return an error if not. Automatically uses the pre-cached tph_max_type
+ * value, the negotiated maximum requester type between device and root port.
  *
  * Return: 0 on success, otherwise negative value (-errno)
  */
@@ -428,7 +427,6 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 {
 	u32 reg;
 	u8 dev_modes;
-	u8 rp_req_type;
 
 	/* Honor "notph" kernel parameter */
 	if (pci_tph_disabled)
@@ -440,6 +438,9 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 	if (pdev->tph_enabled)
 		return -EBUSY;
 
+	if (pdev->tph_max_type == PCI_TPH_REQ_DISABLE)
+		return -EINVAL;
+
 	/* Sanitize and check ST mode compatibility */
 	mode &= PCI_TPH_CTRL_MODE_SEL_MASK;
 	dev_modes = get_st_modes(pdev);
@@ -448,23 +449,7 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 
 	pdev->tph_mode = mode;
 
-	/* Get req_type supported by device and its Root Port */
-	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
-	if (FIELD_GET(PCI_TPH_CAP_EXT_TPH, reg))
-		pdev->tph_req_type = PCI_TPH_REQ_EXT_TPH;
-	else
-		pdev->tph_req_type = PCI_TPH_REQ_TPH_ONLY;
-
-	/* Check if the device is behind a Root Port */
-	if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) {
-		rp_req_type = get_rp_completer_type(pdev);
-
-		/* Final req_type is the smallest value of two */
-		pdev->tph_req_type = min(pdev->tph_req_type, rp_req_type);
-	}
-
-	if (pdev->tph_req_type == PCI_TPH_REQ_DISABLE)
-		return -EINVAL;
+	pdev->tph_req_type = pdev->tph_max_type;
 
 	/* Write them into TPH control register */
 	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CTRL, &reg);
@@ -551,13 +536,31 @@ void pci_no_tph(void)
 
 void pci_tph_init(struct pci_dev *pdev)
 {
+	u8 tph_req_type, rp_req_type;
 	int num_entries;
 	u32 save_size;
+	u32 reg = 0;
 
 	pdev->tph_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_TPH);
 	if (!pdev->tph_cap)
 		return;
 
+	/* Get req_type supported by device and its Root Port */
+	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
+	if (FIELD_GET(PCI_TPH_CAP_EXT_TPH, reg))
+		tph_req_type = PCI_TPH_REQ_EXT_TPH;
+	else
+		tph_req_type = PCI_TPH_REQ_TPH_ONLY;
+
+	/* Check if the device is behind a Root Port */
+	if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) {
+		rp_req_type = get_rp_completer_type(pdev);
+		/* Final req_type is the smallest value of two */
+		tph_req_type = min(tph_req_type, rp_req_type);
+	}
+
+	pdev->tph_max_type = tph_req_type;
+
 	num_entries = pcie_tph_get_st_table_size(pdev);
 	save_size = sizeof(u32) + num_entries * sizeof(u16);
 	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index ce520523b152..5ed58799a9b1 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -587,7 +587,9 @@ struct pci_dev {
 	u8 reset_methods[PCI_NUM_RESET_METHODS]; /* In priority order */
 
 #ifdef CONFIG_PCIE_TPH
-	u16		tph_cap:15;	/* TPH capability offset */
+	u16		tph_cap:13;	/* TPH capability offset */
+	u16		tph_max_type:2; /* Negotiated max TPH requester type
+					   between device and root port */
 	u16		tph_enabled:1;	/* Whether TPH is enabled */
 	u8		tph_mode;	/* TPH mode */
 	u8		tph_req_type;	/* TPH requester type */
-- 
2.17.1

[PATCH v20 07/16] PCI/TPH: Refactor pcie_enable_tph and add requester variant

[Chengwen Feng]

Refactor the core TPH enable logic by renaming the original function to
pcie_enable_tph_ext(), which accepts an explicit requester type argument.

- Keep the original pcie_enable_tph() as a compatibility wrapper; it uses
  the pre-cached tph_max_type negotiated capability automatically, so
  existing drivers calling pcie_enable_tph() require no modification.
- Export new pcie_enable_tph_ext() to support explicit requester type
  selection, intended for use by the upcoming VFIO TPH series.
- Add requester type validation to reject values out of hardware negotiated
  range or invalid reserved types.

Update pci-tph.h header and CONFIG_PCIE_TPH stub inline definitions
to match the new function signatures.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 41 ++++++++++++++++++++++++++++++-----------
 include/linux/pci-tph.h |  8 ++++++--
 2 files changed, 36 insertions(+), 13 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index d123f91775c6..1239a919b9a4 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -409,21 +409,18 @@ void pcie_disable_tph(struct pci_dev *pdev)
 EXPORT_SYMBOL(pcie_disable_tph);
 
 /**
- * pcie_enable_tph - Enable TPH support for device using a specific ST mode
+ * pcie_enable_tph_ext - Enable TPH support with specific ST mode and
+ * requester type
  * @pdev: PCI device
- * @mode: ST mode to enable. Current supported modes include:
- *
- *   - PCI_TPH_ST_NS_MODE: NO ST Mode
- *   - PCI_TPH_ST_IV_MODE: Interrupt Vector Mode
- *   - PCI_TPH_ST_DS_MODE: Device Specific Mode
+ * @mode: ST mode to enable: PCI_TPH_ST_(NS_MODE / IV_MODE / DS_MODE)
+ * @req_type: Requester type to enable: PCI_TPH_REQ_(TPH_ONLY / EXT_TPH)
  *
- * Check whether the mode is actually supported by the device before enabling
- * and return an error if not. Automatically uses the pre-cached tph_max_type
- * value, the negotiated maximum requester type between device and root port.
+ * Check whether the mode and requester type is actually supported by the
+ * device before enabling and return an error if not.
  *
  * Return: 0 on success, otherwise negative value (-errno)
  */
-int pcie_enable_tph(struct pci_dev *pdev, int mode)
+int pcie_enable_tph_ext(struct pci_dev *pdev, u8 mode, u8 req_type)
 {
 	u32 reg;
 	u8 dev_modes;
@@ -441,6 +438,11 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 	if (pdev->tph_max_type == PCI_TPH_REQ_DISABLE)
 		return -EINVAL;
 
+	/* Sanitize and check TPH requester type compatibility */
+	if (req_type == PCI_TPH_REQ_DISABLE || req_type == 2 ||
+		req_type > pdev->tph_max_type)
+		return -EINVAL;
+
 	/* Sanitize and check ST mode compatibility */
 	mode &= PCI_TPH_CTRL_MODE_SEL_MASK;
 	dev_modes = get_st_modes(pdev);
@@ -449,7 +451,7 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 
 	pdev->tph_mode = mode;
 
-	pdev->tph_req_type = pdev->tph_max_type;
+	pdev->tph_req_type = req_type;
 
 	/* Write them into TPH control register */
 	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CTRL, &reg);
@@ -463,6 +465,23 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 
 	return 0;
 }
+EXPORT_SYMBOL(pcie_enable_tph_ext);
+
+/**
+ * pcie_enable_tph - Enable TPH support for device using a specific ST mode
+ * @pdev: PCI device
+ * @mode: ST mode to enable: PCI_TPH_ST_(NS_MODE / IV_MODE / DS_MODE)
+ *
+ * Wrapper for pcie_enable_tph_ext(), automatically uses the pre-cached
+ * tph_max_type value (negotiated maximum requester type between device and
+ * root port).
+ *
+ * Return: 0 on success, otherwise negative value (-errno)
+ */
+int pcie_enable_tph(struct pci_dev *pdev, u8 mode)
+{
+	return pcie_enable_tph_ext(pdev, mode, pdev->tph_max_type);
+}
 EXPORT_SYMBOL(pcie_enable_tph);
 
 void pci_restore_tph_state(struct pci_dev *pdev)
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 0e6aa1895050..3055776495c1 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -29,7 +29,8 @@ int pcie_tph_get_cpu_st(struct pci_dev *dev,
 			enum tph_mem_type mem_type,
 			unsigned int cpu, u16 *tag);
 void pcie_disable_tph(struct pci_dev *pdev);
-int pcie_enable_tph(struct pci_dev *pdev, int mode);
+int pcie_enable_tph_ext(struct pci_dev *pdev, u8 mode, u8 req_type);
+int pcie_enable_tph(struct pci_dev *pdev, u8 mode);
 u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
 u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
@@ -43,7 +44,10 @@ static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 				      unsigned int cpu, u16 *tag)
 { return -EINVAL; }
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
-static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
+static inline int pcie_enable_tph_ext(struct pci_dev *pdev, u8 mode,
+				      u8 req_type)
+{ return -EINVAL; }
+static inline int pcie_enable_tph(struct pci_dev *pdev, u8 mode)
 { return -EINVAL; }
 static inline u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
 { return 0; }
-- 
2.17.1

[PATCH v20 08/16] PCI/TPH: Refactor pcie_tph_get_cpu_st & add requester variant

[Chengwen Feng]

Refactor pcie_tph_get_cpu_st() and add pcie_tph_get_cpu_st_ext() extended
variant with explicit requester type parameter.

- Preserve original pcie_tph_get_cpu_st() as a compatibility wrapper,
  using the default negotiated tph_max_type automatically, so existing
  callers require no modification.
- Add exported pcie_tph_get_cpu_st_ext() to accept an explicit requester
  type argument for ST namespace selection, used by upcoming VFIO TPH
  support.
- Add requester type validation to reject invalid values and enforce the
  hardware negotiated tph_max_type capability limit.

Update pci-tph.h header and CONFIG_PCIE_TPH stub inline definitions
to match the new function signatures.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 45 ++++++++++++++++++++++++++++++++---------
 include/linux/pci-tph.h | 10 +++++++--
 2 files changed, 44 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 1239a919b9a4..633dc6308b68 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -276,20 +276,22 @@ static int write_tag_to_st_table(struct pci_dev *pdev, int index, u16 tag)
 }
 
 /**
- * pcie_tph_get_cpu_st() - Retrieve Steering Tag for a target memory associated
- * with a specific CPU
+ * pcie_tph_get_cpu_st_ext() - Retrieve Steering Tag for a target memory
+ * associated with a specific CPU, with explicit requester type selection
  * @pdev: PCI device
  * @mem_type: target memory type (volatile or persistent RAM)
+ * @req_type: TPH requester type to select ST namespace
  * @cpu: associated CPU id
  * @tag: Steering Tag to be returned
  *
- * Return the Steering Tag for a target memory that is associated with a
- * specific CPU as indicated by cpu.
+ * Return the Steering Tag for a target memory associated with a specific CPU,
+ * using the specified TPH requester type to select the correct ST namespace.
+ * Validates requester type against hardware negotiated capability limits.
  *
  * Return: 0 if success, otherwise negative value (-errno)
  */
-int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
-			unsigned int cpu, u16 *tag)
+int pcie_tph_get_cpu_st_ext(struct pci_dev *pdev, enum tph_mem_type mem_type,
+			    u8 req_type, unsigned int cpu, u16 *tag)
 {
 #ifdef CONFIG_ACPI
 	struct pci_dev *rp;
@@ -298,6 +300,10 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
 	u32 cpu_uid;
 	int ret;
 
+	if (req_type == PCI_TPH_REQ_DISABLE || req_type == 2 ||
+		req_type > pdev->tph_max_type)
+		return -EINVAL;
+
 	ret = acpi_get_cpu_uid(cpu, &cpu_uid);
 	if (ret != 0)
 		return ret;
@@ -313,17 +319,38 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
 		return -EINVAL;
 	}
 
-	*tag = tph_extract_tag(mem_type, pdev->tph_req_type, &info);
+	*tag = tph_extract_tag(mem_type, req_type, &info);
 
-	pci_dbg(pdev, "get steering tag: mem_type=%s, cpu=%d, tag=%#04x\n",
+	pci_dbg(pdev, "get steering tag: mem_type=%s, req_type=%u, cpu=%d, tag=%#04x\n",
 		(mem_type == TPH_MEM_TYPE_VM) ? "volatile" : "persistent",
-		cpu, *tag);
+		req_type, cpu, *tag);
 
 	return 0;
 #else
 	return -ENODEV;
 #endif
 }
+EXPORT_SYMBOL(pcie_tph_get_cpu_st_ext);
+
+/**
+ * pcie_tph_get_cpu_st() - Retrieve Steering Tag for a target memory associated
+ * with a specific CPU
+ * @pdev: PCI device
+ * @mem_type: target memory type (volatile or persistent RAM)
+ * @cpu: associated CPU id
+ * @tag: Steering Tag to be returned
+ *
+ * Return the Steering Tag for a target memory associated with a specific CPU,
+ * using the default negotiated TPH requester type.
+ *
+ * Return: 0 if success, otherwise negative value (-errno)
+ */
+int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
+			unsigned int cpu, u16 *tag)
+{
+	return pcie_tph_get_cpu_st_ext(pdev, mem_type, pdev->tph_max_type,
+				       cpu, tag);
+}
 EXPORT_SYMBOL(pcie_tph_get_cpu_st);
 
 /**
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 3055776495c1..6901f6e2f8c7 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -25,8 +25,9 @@ enum tph_mem_type {
 #ifdef CONFIG_PCIE_TPH
 int pcie_tph_set_st_entry(struct pci_dev *pdev,
 			  unsigned int index, u16 tag);
-int pcie_tph_get_cpu_st(struct pci_dev *dev,
-			enum tph_mem_type mem_type,
+int pcie_tph_get_cpu_st_ext(struct pci_dev *dev, enum tph_mem_type mem_type,
+			    u8 req_type, unsigned int cpu, u16 *tag);
+int pcie_tph_get_cpu_st(struct pci_dev *dev, enum tph_mem_type mem_type,
 			unsigned int cpu, u16 *tag);
 void pcie_disable_tph(struct pci_dev *pdev);
 int pcie_enable_tph_ext(struct pci_dev *pdev, u8 mode, u8 req_type);
@@ -39,6 +40,11 @@ u8 pcie_tph_completer_type(struct pci_dev *pdev);
 static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
 					unsigned int index, u16 tag)
 { return -EINVAL; }
+static inline int pcie_tph_get_cpu_st_ext(struct pci_dev *dev,
+					  enum tph_mem_type mem_type,
+					  u8 req_type, unsigned int cpu,
+					  u16 *tag)
+{ return -EINVAL; }
 static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 				      enum tph_mem_type mem_type,
 				      unsigned int cpu, u16 *tag)
-- 
2.17.1

[PATCH v20 09/16] PCI/TPH: Add macros to check standard and extended TPH support

[Chengwen Feng]

Add two helper macros for TPH capability checking:
- pcie_std_tph_supported(): check basic standard TPH support
- pcie_ext_tph_supported(): check extended TPH requester support

They rely on the cached tph_max_type value validated during probe,
and provide consistent capability queries for subsequent VFIO TPH
virtualization code. Stub definitions are added for !CONFIG_PCIE_TPH.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 include/linux/pci-tph.h | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 6901f6e2f8c7..04b521b7c2c7 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -9,7 +9,7 @@
 #ifndef LINUX_PCI_TPH_H
 #define LINUX_PCI_TPH_H
 
-#include <linux/pci_regs.h>
+#include <linux/pci.h>
 
 /*
  * According to the ECN for PCI Firmware Spec, Steering Tag can be different
@@ -23,6 +23,18 @@ enum tph_mem_type {
 };
 
 #ifdef CONFIG_PCIE_TPH
+/**
+ * pcie_std_tph_supported - check standard TPH requester support
+ * @pdev: PCI device
+ */
+#define pcie_std_tph_supported(pdev) \
+	((pdev)->tph_max_type >= PCI_TPH_REQ_TPH_ONLY)
+/**
+ * pcie_ext_tph_supported - check extended TPH requester support
+ * @pdev: PCI device
+ */
+#define pcie_ext_tph_supported(pdev) \
+	((pdev)->tph_max_type == PCI_TPH_REQ_EXT_TPH)
 int pcie_tph_set_st_entry(struct pci_dev *pdev,
 			  unsigned int index, u16 tag);
 int pcie_tph_get_cpu_st_ext(struct pci_dev *dev, enum tph_mem_type mem_type,
@@ -37,6 +49,8 @@ u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
 u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
 u8 pcie_tph_completer_type(struct pci_dev *pdev);
 #else
+#define pcie_std_tph_supported(pdev) false
+#define pcie_ext_tph_supported(pdev) false
 static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
 					unsigned int index, u16 tag)
 { return -EINVAL; }
-- 
2.17.1

[PATCH v20 10/16] PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming helper

[Chengwen Feng]

Add batch variant pcie_tph_set_st_entries() to write a contiguous range
of ST table entries, returning the number of successfully written entries
or negative errno.
- Keep original pcie_tph_set_st_entry() as a wrapper for backward
  compatibility
- Disable/enable TPH once per batch instead of per entry for better
  performance
- Add header declaration and fallback static inline version
- Add error logging for failed ST entry writes

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 74 +++++++++++++++++++++++++----------------
 include/linux/pci-tph.h |  6 ++++
 2 files changed, 51 insertions(+), 29 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 633dc6308b68..8194f8b45e3c 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -354,21 +354,19 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
 EXPORT_SYMBOL(pcie_tph_get_cpu_st);
 
 /**
- * pcie_tph_set_st_entry() - Set Steering Tag in the ST table entry
+ * pcie_tph_set_st_entries - Batch set Steering Tag entries in the ST table
  * @pdev: PCI device
- * @index: ST table entry index
- * @tag: Steering Tag to be written
- *
- * Figure out the proper location of ST table, either in the MSI-X table or
- * in the TPH Extended Capability space, and write the Steering Tag into
- * the ST entry pointed by index.
+ * @start: starting ST table index
+ * @count: number of contiguous entries to write
+ * @tags: array of u16 tag values to program
  *
- * Return: 0 if success, otherwise negative value (-errno)
+ * Return: number of successfully written entries, or negative errno
  */
-int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
+int pcie_tph_set_st_entries(struct pci_dev *pdev, unsigned int start,
+			    unsigned int count, const u16 *tags)
 {
-	u32 loc;
 	int err = 0;
+	u32 loc, i;
 
 	if (!pdev->tph_cap)
 		return -EINVAL;
@@ -378,7 +376,7 @@ int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
 
 	/* No need to write tag if device is in "No ST Mode" */
 	if (pdev->tph_mode == PCI_TPH_ST_NS_MODE)
-		return 0;
+		return count;
 
 	/*
 	 * Disable TPH before updating ST to avoid potential instability as
@@ -387,29 +385,47 @@ int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
 	set_ctrl_reg_req_en(pdev, PCI_TPH_REQ_DISABLE);
 
 	loc = pcie_tph_get_st_table_loc(pdev);
-
-	switch (loc) {
-	case PCI_TPH_LOC_MSIX:
-		err = pci_msix_write_tph_tag(pdev, index, tag);
-		break;
-	case PCI_TPH_LOC_CAP:
-		err = write_tag_to_st_table(pdev, index, tag);
-		break;
-	default:
-		err = -EINVAL;
-	}
-
-	if (err) {
-		pcie_disable_tph(pdev);
-		return err;
+	for (i = 0; i < count; i++) {
+		switch (loc) {
+		case PCI_TPH_LOC_MSIX:
+			err = pci_msix_write_tph_tag(pdev, start + i, tags[i]);
+			break;
+		case PCI_TPH_LOC_CAP:
+			err = write_tag_to_st_table(pdev, start + i, tags[i]);
+			break;
+		default:
+			err = -EINVAL;
+		}
+		if (err != 0) {
+			pci_err(pdev, "failed to set steering tag: %s table, index=%u, tags=%u, ret=%d\n",
+				(loc == PCI_TPH_LOC_MSIX) ? "MSI-X" : "ST",
+				start + i, tags[i], err);
+			pcie_disable_tph(pdev);
+			return i > 0 ? i : err;
+		}
 	}
 
 	set_ctrl_reg_req_en(pdev, pdev->tph_req_type);
 
-	pci_dbg(pdev, "set steering tag: %s table, index=%d, tag=%#04x\n",
-		(loc == PCI_TPH_LOC_MSIX) ? "MSI-X" : "ST", index, tag);
+	pci_dbg(pdev, "batch set steering tag: %s table, start=%u, count=%u success!\n",
+		(loc == PCI_TPH_LOC_MSIX) ? "MSI-X" : "ST", start, count);
 
-	return 0;
+	return count;
+}
+EXPORT_SYMBOL(pcie_tph_set_st_entries);
+
+/**
+ * pcie_tph_set_st_entry() - Set Steering Tag in the ST table entry
+ * @pdev: PCI device
+ * @index: ST table entry index
+ * @tag: Steering Tag to be written
+ *
+ * Return: 0 if success, otherwise negative value (-errno)
+ */
+int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
+{
+	int ret = pcie_tph_set_st_entries(pdev, index, 1, &tag);
+	return ret == 1 ? 0 : ret;
 }
 EXPORT_SYMBOL(pcie_tph_set_st_entry);
 
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 04b521b7c2c7..c26bf8bba478 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -37,6 +37,8 @@ enum tph_mem_type {
 	((pdev)->tph_max_type == PCI_TPH_REQ_EXT_TPH)
 int pcie_tph_set_st_entry(struct pci_dev *pdev,
 			  unsigned int index, u16 tag);
+int pcie_tph_set_st_entries(struct pci_dev *pdev, unsigned int start,
+			    unsigned int count, const u16 *tags);
 int pcie_tph_get_cpu_st_ext(struct pci_dev *dev, enum tph_mem_type mem_type,
 			    u8 req_type, unsigned int cpu, u16 *tag);
 int pcie_tph_get_cpu_st(struct pci_dev *dev, enum tph_mem_type mem_type,
@@ -54,6 +56,10 @@ u8 pcie_tph_completer_type(struct pci_dev *pdev);
 static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
 					unsigned int index, u16 tag)
 { return -EINVAL; }
+static inline int pcie_tph_set_st_entries(struct pci_dev *pdev,
+					  unsigned int start,
+					  unsigned int count, const u16 *tags)
+{ return -EINVAL; }
 static inline int pcie_tph_get_cpu_st_ext(struct pci_dev *dev,
 					  enum tph_mem_type mem_type,
 					  u8 req_type, unsigned int cpu,
-- 
2.17.1

[PATCH v20 11/16] PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM

[Chengwen Feng]

Add new helper pci_tph_dsm_supported() to check whether the root port
of a given PCI device exposes the TPH ST ACPI _DSM method.

Wrap all ACPI logic under CONFIG_ACPI guard, return false directly
when ACPI is disabled.

This helper is prepared for follow-up VFIO TPH virtualization patches
to uniformly judge platform CPU TPH resolving support.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 28 ++++++++++++++++++++++++++++
 include/linux/pci-tph.h |  3 +++
 2 files changed, 31 insertions(+)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 8194f8b45e3c..e6a850ab2b03 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -627,3 +627,31 @@ void pci_tph_init(struct pci_dev *pdev)
 	save_size = sizeof(u32) + num_entries * sizeof(u16);
 	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);
 }
+
+/**
+ * pcie_tph_dsm_supported - Check if root port exposes TPH ST _DSM
+ * @pdev: target PCI device
+ *
+ * Return true if root port of @pdev provides TPH ST DSM function,
+ * false otherwise (ACPI disabled or DSM missing).
+ */
+bool pcie_tph_dsm_supported(struct pci_dev *pdev)
+{
+#ifdef CONFIG_ACPI
+	struct pci_dev *rp = pcie_find_root_port(pdev);
+	acpi_handle handle;
+
+	if (!rp || !rp->bus || !rp->bus->bridge)
+		return false;
+
+	handle = ACPI_HANDLE(rp->bus->bridge);
+	if (!handle)
+		return false;
+
+	return acpi_check_dsm(handle, &pci_acpi_dsm_guid, 7,
+			      BIT(TPH_ST_DSM_FUNC_INDEX));
+#else
+	return false;
+#endif
+}
+EXPORT_SYMBOL_GPL(pcie_tph_dsm_supported);
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index c26bf8bba478..aeaf701e7965 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -50,6 +50,7 @@ u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
 u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
 u8 pcie_tph_completer_type(struct pci_dev *pdev);
+bool pcie_tph_dsm_supported(struct pci_dev *pdev);
 #else
 #define pcie_std_tph_supported(pdev) false
 #define pcie_ext_tph_supported(pdev) false
@@ -83,6 +84,8 @@ static inline u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
 { return PCI_TPH_REQ_DISABLE; }
 static inline u8 pcie_tph_completer_type(struct pci_dev *pdev)
 { return PCI_EXP_DEVCAP2_TPH_COMP_NONE; }
+static inline bool pcie_tph_dsm_supported(struct pci_dev *pdev)
+{ return false; }
 #endif
 
 #endif /* LINUX_PCI_TPH_H */
-- 
2.17.1

[PATCH v20 12/16] vfio/pci: Add basic NO-ST TPH VFIO device feature support

[Chengwen Feng]

Implement basic PCIe TPH support in NO-ST mode:
- Add tph_policy module parameter, currently locked to NO-ST mode
  * Return DMABUF TPH PH value via VFIO resolve interface only
  * Virtualize TPH capability register to only advertise NO-ST capabilities
  * Allow TPH control register writes to enable NO-ST TPH mode
- Add tph_opt_in guard: all TPH operations require prior opt-in via
  VFIO_DEVICE_FEATURE_TPH SET ioctl
- Add VFIO_DEVICE_FEATURE_TPH (opt-in + capability query)
- Add VFIO_DEVICE_FEATURE_TPH_RESOLVE (DMABUF processing hint resolve only)
- Add TPH extended capability virtual read/write handlers, hide raw ST
  fields, ST table reads return zero by default

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci.c        |  19 ++++
 drivers/vfio/pci/vfio_pci_config.c | 158 +++++++++++++++++++++++++----
 drivers/vfio/pci/vfio_pci_core.c   | 126 +++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h   |   4 +
 include/linux/vfio_pci_core.h      |   3 +
 include/uapi/linux/vfio.h          |  54 ++++++++++
 6 files changed, 347 insertions(+), 17 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 830369ff878d..1612b109a30a 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -60,6 +60,22 @@ static bool disable_denylist;
 module_param(disable_denylist, bool, 0444);
 MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users.");
 
+static unsigned int tph_policy;
+static int tph_policy_set(const char *val, const struct kernel_param *kp)
+{
+	return param_set_uint_minmax(val, kp, VFIO_PCI_TPH_POLICY_NO_ST,
+				     VFIO_PCI_TPH_POLICY_NO_ST);
+}
+static const struct kernel_param_ops tph_param_ops = {
+	.set = tph_policy_set,
+	.get = param_get_uint,
+};
+module_param_cb(tph_policy, &tph_param_ops, &tph_policy, 0644);
+MODULE_PARM_DESC(tph_policy,
+	"Global TPH policy level (0=default No-ST):\n"
+	"0 = No-ST mode: RESOLVE only returns PH for DMABUF source;\n"
+	"                ST programming unavailable\n");
+
 static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev)
 {
 	switch (pdev->vendor) {
@@ -142,6 +158,9 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev)
 #ifdef CONFIG_VFIO_PCI_VGA
 	vdev->disable_vga = disable_vga;
 #endif
+#ifdef CONFIG_PCIE_TPH
+	vdev->tph_policy = tph_policy;
+#endif
 
 	return vfio_pci_core_init_dev(core_vdev);
 }
diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index a10ed733f0e3..8bdac647397c 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -22,6 +22,7 @@
 
 #include <linux/fs.h>
 #include <linux/pci.h>
+#include <linux/pci-tph.h>
 #include <linux/uaccess.h>
 #include <linux/vfio.h>
 #include <linux/slab.h>
@@ -1085,6 +1086,140 @@ static int __init init_pci_ext_cap_pwr_perm(struct perm_bits *perm)
 	return 0;
 }
 
+static int vfio_find_cap_start(struct vfio_pci_core_device *vdev, int pos)
+{
+	u8 cap;
+	int base = (pos >= PCI_CFG_SPACE_SIZE) ? PCI_CFG_SPACE_SIZE :
+						 PCI_STD_HEADER_SIZEOF;
+	cap = vdev->pci_config_map[pos];
+
+	if (cap == PCI_CAP_ID_BASIC)
+		return 0;
+
+	/* XXX Can we have to abutting capabilities of the same type? */
+	while (pos - 1 >= base && vdev->pci_config_map[pos - 1] == cap)
+		pos--;
+
+	return pos;
+}
+
+static void vfio_update_tph_vconfig_bytes(struct vfio_pci_core_device *vdev,
+					  int pos)
+{
+	__le32 *vptr = (__le32 *)&vdev->vconfig[pos + PCI_TPH_CAP];
+	struct pci_dev *pdev = vdev->pdev;
+	u32 val = le32_to_cpu(*vptr);
+
+	if (!pcie_ext_tph_supported(pdev)) {
+		/* Remove extended TPH if root-port doesn't support */
+		val &= ~PCI_TPH_CAP_EXT_TPH;
+	}
+
+	if (vdev->tph_policy == VFIO_PCI_TPH_POLICY_NO_ST) {
+		/* Report only No-ST mode supported */
+		val &= ~(PCI_TPH_CAP_ST_IV | PCI_TPH_CAP_ST_DS |
+			 PCI_TPH_CAP_LOC_MASK | PCI_TPH_CAP_ST_MASK);
+	}
+
+	*vptr = cpu_to_le32(val);
+
+	/* Backup updated capability register value */
+	vdev->tph_cap_virt = val;
+}
+
+/* Permissions for TPH extended capability */
+static int __init init_pci_ext_cap_tph_perm(struct perm_bits *perm)
+{
+	/*
+	 * We only virtualize the TPH header (excluding the ST table).
+	 * For the ST table (if any), the read operation returns 0 by
+	 * default, and the write operation is discarded by default.
+	 */
+	if (alloc_perm_bits(perm, PCI_TPH_BASE_SIZEOF))
+		return -ENOMEM;
+
+	p_setd(perm, 0, ALL_VIRT, NO_WRITE);
+	p_setd(perm, PCI_TPH_CAP, ALL_VIRT, NO_WRITE);
+
+	p_setd(perm, PCI_TPH_CTRL, ALL_VIRT,
+	       PCI_TPH_CTRL_MODE_SEL_MASK | PCI_TPH_CTRL_REQ_EN_MASK);
+
+	return 0;
+}
+
+static int vfio_tph_config_read(struct vfio_pci_core_device *vdev, int pos,
+				int count, struct perm_bits *perm,
+				int offset, __le32 *val)
+{
+	if (offset >= PCI_TPH_BASE_SIZEOF) {
+		*val = cpu_to_le32(0);
+		return count;
+	}
+
+	if (offset >= PCI_TPH_CAP && !vdev->tph_opt_in) {
+		*val = cpu_to_le32(0);
+		return count;
+	}
+
+	return vfio_default_config_read(vdev, pos, count, perm, offset, val);
+}
+
+static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
+				 int count, struct perm_bits *perm,
+				 int offset, __le32 val)
+{
+	u16 start = vfio_find_cap_start(vdev, pos);
+	struct pci_dev *pdev = vdev->pdev;
+	u32 org_ctrl, new_ctrl, cap;
+	u8 mode, req, org_req;
+	__le32 org_val = 0;
+	u16 cmd;
+	int ret;
+
+	if (!vdev->tph_opt_in || offset >= PCI_TPH_BASE_SIZEOF)
+		return count;
+
+	cmd = vfio_pci_memory_lock_and_enable(vdev);
+
+	org_ctrl = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CTRL]);
+	vfio_default_config_read(vdev, pos, count, perm, offset, &org_val);
+
+	ret = vfio_default_config_write(vdev, pos, count, perm, offset, val);
+	if (ret != count)
+		goto out;
+
+	new_ctrl = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CTRL]);
+	if (new_ctrl == org_ctrl)
+		goto out; /* Only care about changes in TPH_CTRL. */
+
+	cap = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CAP]);
+	mode = FIELD_GET(PCI_TPH_CTRL_MODE_SEL_MASK, new_ctrl);
+	req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, new_ctrl);
+	if (mode != PCI_TPH_ST_NS_MODE || !(cap & (1u << mode)) || req == 0x2 ||
+		(req == PCI_TPH_REQ_EXT_TPH && !(cap & PCI_TPH_CAP_EXT_TPH)))
+		goto restore; /* Drop invalid or unsupported write value */
+
+	org_req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, org_ctrl);
+	if (req == org_req)
+		goto restore; /* Only care about requester enable */
+
+	if (req == PCI_TPH_REQ_TPH_ONLY || req == PCI_TPH_REQ_EXT_TPH) {
+		ret = pcie_enable_tph_ext(pdev, mode, req);
+		if (ret)
+			goto restore;
+	} else if (req == PCI_TPH_REQ_DISABLE) {
+		pcie_disable_tph(vdev->pdev);
+	}
+
+	goto out;
+
+restore:
+	vfio_default_config_write(vdev, pos, count, perm, offset, org_val);
+out:
+	vfio_pci_memory_unlock_and_restore(vdev, cmd);
+	return count;
+}
+
 /*
  * Initialize the shared permission tables
  */
@@ -1100,6 +1235,7 @@ void vfio_pci_uninit_perm_bits(void)
 
 	free_perm_bits(&ecap_perms[PCI_EXT_CAP_ID_ERR]);
 	free_perm_bits(&ecap_perms[PCI_EXT_CAP_ID_PWR]);
+	free_perm_bits(&ecap_perms[PCI_EXT_CAP_ID_TPH]);
 }
 
 int __init vfio_pci_init_perm_bits(void)
@@ -1120,6 +1256,9 @@ int __init vfio_pci_init_perm_bits(void)
 	/* Extended capabilities */
 	ret |= init_pci_ext_cap_err_perm(&ecap_perms[PCI_EXT_CAP_ID_ERR]);
 	ret |= init_pci_ext_cap_pwr_perm(&ecap_perms[PCI_EXT_CAP_ID_PWR]);
+	ret |= init_pci_ext_cap_tph_perm(&ecap_perms[PCI_EXT_CAP_ID_TPH]);
+	ecap_perms[PCI_EXT_CAP_ID_TPH].readfn = vfio_tph_config_read;
+	ecap_perms[PCI_EXT_CAP_ID_TPH].writefn = vfio_tph_config_write;
 	ecap_perms[PCI_EXT_CAP_ID_VNDR].writefn = vfio_raw_config_write;
 	ecap_perms[PCI_EXT_CAP_ID_DVSEC].writefn = vfio_raw_config_write;
 
@@ -1129,23 +1268,6 @@ int __init vfio_pci_init_perm_bits(void)
 	return ret;
 }
 
-static int vfio_find_cap_start(struct vfio_pci_core_device *vdev, int pos)
-{
-	u8 cap;
-	int base = (pos >= PCI_CFG_SPACE_SIZE) ? PCI_CFG_SPACE_SIZE :
-						 PCI_STD_HEADER_SIZEOF;
-	cap = vdev->pci_config_map[pos];
-
-	if (cap == PCI_CAP_ID_BASIC)
-		return 0;
-
-	/* XXX Can we have to abutting capabilities of the same type? */
-	while (pos - 1 >= base && vdev->pci_config_map[pos - 1] == cap)
-		pos--;
-
-	return pos;
-}
-
 static int vfio_msi_config_read(struct vfio_pci_core_device *vdev, int pos,
 				int count, struct perm_bits *perm,
 				int offset, __le32 *val)
@@ -1701,6 +1823,8 @@ static int vfio_ecap_init(struct vfio_pci_core_device *vdev)
 		ret = vfio_fill_vconfig_bytes(vdev, epos, len);
 		if (ret)
 			return ret;
+		if (ecap == PCI_EXT_CAP_ID_TPH)
+			vfio_update_tph_vconfig_bytes(vdev, epos);
 
 		/*
 		 * If we're just using this capability to anchor the list,
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 4e5e34a77b76..82582c8ef69d 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -13,6 +13,8 @@
 #include <linux/aperture.h>
 #include <linux/debugfs.h>
 #include <linux/device.h>
+#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
 #include <linux/eventfd.h>
 #include <linux/file.h>
 #include <linux/interrupt.h>
@@ -30,6 +32,7 @@
 #include <linux/sched/mm.h>
 #include <linux/iommufd.h>
 #include <linux/pci-p2pdma.h>
+#include <linux/pci-tph.h>
 #include <linux/seq_file.h>
 #if IS_ENABLED(CONFIG_EEH)
 #include <asm/eeh.h>
@@ -610,6 +613,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 		goto out_disable_device;
 
 	vdev->reset_works = !ret;
+	vdev->tph_opt_in = false;
 	pci_save_state(pdev);
 	vdev->pci_saved_state = pci_store_saved_state(pdev);
 	if (!vdev->pci_saved_state)
@@ -1607,6 +1611,123 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
 	return 0;
 }
 
+static u32 vfio_pci_get_tph_capability(struct vfio_pci_core_device *vdev)
+{
+	return VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH;
+}
+
+static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
+				     u32 flags,
+				     struct vfio_device_feature_tph __user *arg,
+				     size_t argsz)
+{
+	struct vfio_device_feature_tph tph;
+	int ret;
+
+	if (!pcie_std_tph_supported(vdev->pdev))
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz,
+			VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_SET,
+			sizeof(tph));
+	if (ret != 1)
+		return ret;
+
+	if (copy_from_user(&tph, arg, sizeof(tph)))
+		return -EFAULT;
+
+	if (flags & VFIO_DEVICE_FEATURE_SET) {
+		if (tph.flags != 0)
+			return -EINVAL;
+		vdev->tph_opt_in = 1;
+		return 0;
+	}
+
+	if (!vdev->tph_opt_in)
+		return -EINVAL;
+
+	tph.flags = vfio_pci_get_tph_capability(vdev);
+
+	return copy_to_user(arg, &tph, sizeof(tph)) ? -EFAULT : 0;
+}
+
+static u32 vfio_pci_get_tph_resolve_allow_src(struct vfio_pci_core_device *vdev)
+{
+	u32 flags = vfio_pci_get_tph_capability(vdev);
+	u32 allow_src = 0;
+
+	if (flags & VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH)
+		allow_src |= VFIO_DEVICE_TPH_SRC_DMABUF;
+
+	return allow_src;
+}
+
+static int vfio_pci_get_dmabuf_tph(int fd, bool extended, u16 *st, u8 *ph)
+{
+	struct dma_buf *dmabuf;
+	int ret;
+
+	dmabuf = dma_buf_get(fd);
+	if (IS_ERR(dmabuf))
+		return PTR_ERR(dmabuf);
+
+	ret = dma_resv_lock_interruptible(dmabuf->resv, NULL);
+	if (ret == 0)
+		ret = dma_buf_get_pci_tph(dmabuf, extended, st, ph);
+	dma_resv_unlock(dmabuf->resv);
+
+	dma_buf_put(dmabuf);
+	return ret;
+}
+
+static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
+			u32 flags,
+			struct vfio_device_feature_tph_resolve __user *arg,
+			size_t argsz)
+{
+	struct vfio_device_feature_tph_resolve resolve;
+	u32 user_flags, allow_src;
+	bool extended;
+	u16 tag = 0;
+	u8 ph = 0;
+	int ret;
+
+	if (!vdev->tph_opt_in)
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET,
+				 sizeof(resolve));
+	if (ret != 1)
+		return ret;
+
+	if (copy_from_user(&resolve, arg, sizeof(resolve)))
+		return -EFAULT;
+
+	user_flags = resolve.flags;
+	extended = !!(user_flags & VFIO_DEVICE_TPH_EXTENDED);
+	if (extended && !pcie_ext_tph_supported(vdev->pdev))
+		return -EOPNOTSUPP;
+	user_flags &= ~VFIO_DEVICE_TPH_EXTENDED;
+
+	allow_src = vfio_pci_get_tph_resolve_allow_src(vdev);
+	if (user_flags & ~allow_src || !is_power_of_2(user_flags))
+		return -EINVAL;
+
+	resolve.valid = 0;
+	resolve.ph = 0;
+
+	if (user_flags & VFIO_DEVICE_TPH_SRC_DMABUF) {
+		ret = vfio_pci_get_dmabuf_tph(resolve.src, extended,
+					      &tag, &ph);
+		if (ret)
+			return ret;
+		resolve.ph = ph;
+		resolve.valid = VFIO_DEVICE_TPH_VALID_PH;
+	}
+
+	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
+}
+
 int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 				void __user *arg, size_t argsz)
 {
@@ -1628,6 +1749,11 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 	case VFIO_DEVICE_FEATURE_DMA_BUF_TPH:
 		return vfio_pci_core_feature_dma_buf_tph(vdev, flags, arg,
 							 argsz);
+	case VFIO_DEVICE_FEATURE_TPH:
+		return vfio_pci_core_feature_tph(vdev, flags, arg, argsz);
+	case VFIO_DEVICE_FEATURE_TPH_RESOLVE:
+		return vfio_pci_core_feature_tph_resolve(vdev, flags,
+							 arg, argsz);
 	default:
 		return -ENOTTY;
 	}
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index c58f369be4b3..7fbf938f69e1 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -11,6 +11,10 @@
 /* Cap maximum number of ioeventfds per device (arbitrary) */
 #define VFIO_PCI_IOEVENTFD_MAX		1000
 
+enum vfio_pci_tph_policy {
+	VFIO_PCI_TPH_POLICY_NO_ST = 0,
+};
+
 struct vfio_pci_ioeventfd {
 	struct list_head	next;
 	struct vfio_pci_core_device	*vdev;
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9a1674c152aa..d6064fa9e80a 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -149,6 +149,9 @@ struct vfio_pci_core_device {
 	struct notifier_block	nb;
 	struct rw_semaphore	memory_lock;
 	struct list_head	dmabufs;
+	u8			tph_opt_in;
+	u8			tph_policy;
+	u32			tph_cap_virt;
 };
 
 enum vfio_pci_io_width {
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 4c1c70aac150..ae7a2f083cd5 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -1577,6 +1577,60 @@ struct vfio_device_feature_dma_buf_tph {
 	__u8	ph;
 };
 
+/* PCIe TPH device feature definitions for VFIO_DEVICE_FEATURE ioctl */
+#define VFIO_DEVICE_FEATURE_TPH		14
+#define VFIO_DEVICE_FEATURE_TPH_RESOLVE	15
+
+/*
+ * VFIO_DEVICE_FEATURE_TPH - Control and query PCI TPH capabilities
+ *
+ * SET: Opt-in to TPH feature for the device; flags must be zero.
+ * GET: Return supported TPH capability bits (see VFIO_DEVICE_TPH_CAP_*)
+ *      in flags. Returns error if SET has not been invoked first.
+ *
+ * Userspace must first invoke SET on this feature to enable TPH support,
+ * receive valid capability bits via GET, gain permission for using
+ * VFIO_DEVICE_FEATURE_TPH_RESOLVE, and access the virtualized TPH
+ * capability registers via VFIO config space accesses.
+ */
+struct vfio_device_feature_tph {
+	__u32 flags;
+};
+
+/* DMABUF source resolve processing hint supported */
+#define VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH	(1u << 0)
+
+/*
+ * VFIO_DEVICE_FEATURE_TPH_RESOLVE - Resolve TPH attributes from given source
+ *
+ * GET only
+ *
+ * Requires prior successful SET on VFIO_DEVICE_FEATURE_TPH, otherwise
+ * this feature will return error.
+ *
+ * @flags: IN - source type selector (see VFIO_DEVICE_TPH_SRC_*)
+ *         and steering tag namespace selector (see VFIO_DEVICE_TPH_EXTENDED)
+ * @src:   IN - source identifier, e.g. dmabuf fd when SRC_DMABUF is set in
+ *         flags
+ * @valid: OUT - bitmap indicating valid output fields, see
+ *         VFIO_DEVICE_TPH_VALID_*
+ * @ph:    OUT - TPH processing hint (valid when VALID_PH is set in valid)
+ * @rsv:   Must be zero, reserved for future extensions
+ */
+struct vfio_device_feature_tph_resolve {
+	__u32 flags;
+	__u32 src;
+	__u8  valid;
+#define VFIO_DEVICE_TPH_VALID_PH (1u << 0) /* ph holds valid processing hint */
+	__u8  ph;
+	__u16 rsv;
+};
+
+/* Source holds dma-buf fd */
+#define VFIO_DEVICE_TPH_SRC_DMABUF      (1u << 0)
+/* Use extended 16-bit steering tag namespace */
+#define VFIO_DEVICE_TPH_EXTENDED	(1u << 1)
+
 /* -------- API for Type1 VFIO IOMMU -------- */
 
 /**
-- 
2.17.1

[PATCH v20 13/16] vfio/pci: Add IV-ST TPH policy and VFIO TPH_ST batch programming support

[Chengwen Feng]

Extend VFIO TPH implementation to support IV-ST mode based on the existing
NO-ST baseline, and add VFIO_DEVICE_FEATURE_TPH_ST for batch ST table
programming.

- Add VFIO_PCI_TPH_POLICY_IV_ST policy and extend tph_policy module param
  * NO-ST: hide all ST capabilities, ST programming unavailable
  * IV-ST: expose IV mode + ST location/size, hide only DS capability
- Restrict TPH mode selection writes to NS/IV only, disallow DS
- Report precise TPH_ST capabilities via GET: NONE/DMABUF always available,
  CPU source exposed only if root-port TPH DSM is supported
- Add VFIO_DEVICE_FEATURE_TPH_ST (SET-only) to batch program ST table
  entries supporting NONE, DMABUF, and CPU (volatile/persistent) sources
- Keep TPH_RESOLVE behavior unchanged: only DMABUF source, only return PH
  value
- Enforce tph_opt_in guard and policy check for all ST programming
  operations

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci.c        |   6 +-
 drivers/vfio/pci/vfio_pci_config.c |   8 +-
 drivers/vfio/pci/vfio_pci_core.c   | 151 ++++++++++++++++++++++++++++-
 drivers/vfio/pci/vfio_pci_priv.h   |   1 +
 include/uapi/linux/vfio.h          |  46 ++++++++-
 5 files changed, 206 insertions(+), 6 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 1612b109a30a..02c5c8283099 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -64,7 +64,7 @@ static unsigned int tph_policy;
 static int tph_policy_set(const char *val, const struct kernel_param *kp)
 {
 	return param_set_uint_minmax(val, kp, VFIO_PCI_TPH_POLICY_NO_ST,
-				     VFIO_PCI_TPH_POLICY_NO_ST);
+				     VFIO_PCI_TPH_POLICY_IV_ST);
 }
 static const struct kernel_param_ops tph_param_ops = {
 	.set = tph_policy_set,
@@ -74,7 +74,9 @@ module_param_cb(tph_policy, &tph_param_ops, &tph_policy, 0644);
 MODULE_PARM_DESC(tph_policy,
 	"Global TPH policy level (0=default No-ST):\n"
 	"0 = No-ST mode: RESOLVE only returns PH for DMABUF source;\n"
-	"                ST programming unavailable\n");
+	"                ST programming unavailable\n"
+	"1 = Add IV mode: RESOLVE only returns PH for DMABUF source;\n"
+	"                 ST program supports NONE/DMABUF/CPU sources\n");
 
 static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev)
 {
diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 8bdac647397c..8c68504839b9 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1119,6 +1119,12 @@ static void vfio_update_tph_vconfig_bytes(struct vfio_pci_core_device *vdev,
 		/* Report only No-ST mode supported */
 		val &= ~(PCI_TPH_CAP_ST_IV | PCI_TPH_CAP_ST_DS |
 			 PCI_TPH_CAP_LOC_MASK | PCI_TPH_CAP_ST_MASK);
+	} else if (vdev->tph_policy == VFIO_PCI_TPH_POLICY_IV_ST) {
+		/* Report only No-ST and IV modes supported */
+		val &= ~PCI_TPH_CAP_ST_DS;
+		/* Remove ST location and size if dev doesn't support IV mode */
+		if (!(val & PCI_TPH_CAP_ST_IV))
+			val &= ~(PCI_TPH_CAP_LOC_MASK | PCI_TPH_CAP_ST_MASK);
 	}
 
 	*vptr = cpu_to_le32(val);
@@ -1195,7 +1201,7 @@ static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
 	cap = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CAP]);
 	mode = FIELD_GET(PCI_TPH_CTRL_MODE_SEL_MASK, new_ctrl);
 	req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, new_ctrl);
-	if (mode != PCI_TPH_ST_NS_MODE || !(cap & (1u << mode)) || req == 0x2 ||
+	if (mode > PCI_TPH_ST_IV_MODE || !(cap & (1u << mode)) || req == 0x2 ||
 		(req == PCI_TPH_REQ_EXT_TPH && !(cap & PCI_TPH_CAP_EXT_TPH)))
 		goto restore; /* Drop invalid or unsupported write value */
 
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 82582c8ef69d..df31f4e8766f 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1613,7 +1613,17 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
 
 static u32 vfio_pci_get_tph_capability(struct vfio_pci_core_device *vdev)
 {
-	return VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH;
+	u32 flags = VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH;
+
+	if (vdev->tph_policy >= VFIO_PCI_TPH_POLICY_IV_ST &&
+		(vdev->tph_cap_virt & PCI_TPH_CAP_ST_IV)) {
+		flags |= VFIO_DEVICE_TPH_CAP_ST_NONE;
+		flags |= VFIO_DEVICE_TPH_CAP_ST_DMABUF;
+		if (pcie_tph_dsm_supported(vdev->pdev))
+			flags |= VFIO_DEVICE_TPH_CAP_ST_CPU;
+	}
+
+	return flags;
 }
 
 static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
@@ -1680,6 +1690,32 @@ static int vfio_pci_get_dmabuf_tph(int fd, bool extended, u16 *st, u8 *ph)
 	return ret;
 }
 
+static int vfio_pci_get_tph_st_tag(struct pci_dev *pdev, u32 src_bits,
+				   u32 src_hndl, bool extended, u16 *st)
+{
+	u8 req_type = extended ? PCI_TPH_REQ_EXT_TPH : PCI_TPH_REQ_TPH_ONLY;
+	int ret = 0;
+	u8 ph;
+
+	if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF)
+		ret = vfio_pci_get_dmabuf_tph(src_hndl, extended, st, &ph);
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_VOLATILE)
+		ret = pcie_tph_get_cpu_st_ext(pdev, TPH_MEM_TYPE_VM,
+					      req_type, src_hndl, st);
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT)
+		ret = pcie_tph_get_cpu_st_ext(pdev, TPH_MEM_TYPE_PM,
+					      req_type, src_hndl, st);
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_NONE)
+		*st = 0;
+	else
+		ret = -EINVAL;
+
+	if (ret != 0)
+		*st = 0;
+
+	return ret;
+}
+
 static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
 			u32 flags,
 			struct vfio_device_feature_tph_resolve __user *arg,
@@ -1728,6 +1764,117 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
 	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
 }
 
+static u32 vfio_pci_get_tph_st_allow_src(struct vfio_pci_core_device *vdev)
+{
+	u32 flags = vfio_pci_get_tph_capability(vdev);
+	u32 allow_src = 0;
+
+	if (flags & VFIO_DEVICE_TPH_CAP_ST_DMABUF)
+		allow_src |= VFIO_DEVICE_TPH_SRC_DMABUF;
+	if (flags & VFIO_DEVICE_TPH_CAP_ST_CPU)
+		allow_src |= VFIO_DEVICE_TPH_SRC_CPU_VOLATILE |
+			     VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT;
+	if (flags & VFIO_DEVICE_TPH_CAP_ST_NONE)
+		allow_src |= VFIO_DEVICE_TPH_SRC_NONE;
+
+	return allow_src;
+}
+
+static u32 vfio_pci_get_tph_st_size(struct vfio_pci_core_device *vdev)
+{
+	struct pci_dev *pdev = vdev->pdev;
+	u32 loc = pcie_tph_get_st_table_loc(pdev);
+	int ret;
+
+	if (loc == PCI_TPH_LOC_CAP) {
+		return pcie_tph_get_st_table_size(pdev);
+	} else if (loc == PCI_TPH_LOC_MSIX) {
+		ret = pci_msix_vec_count(pdev);
+		if (ret < 0)
+			return 0;
+		return ret;
+	} else {
+		return 0;
+	}
+}
+
+static int vfio_pci_core_feature_tph_st(struct vfio_pci_core_device *vdev,
+				u32 flags,
+				struct vfio_device_feature_tph_st __user *arg,
+				size_t argsz)
+{
+	struct vfio_device_feature_tph_st tph_st;
+	u32 user_flags, allow_src, st_size;
+	struct pci_dev *pdev = vdev->pdev;
+	bool extended, stop_on_zero;
+	u32 *src_hndl = NULL;
+	u16 *tags = NULL;
+	int ret, i;
+	u16 cmd;
+
+	if (!vdev->tph_opt_in || vdev->tph_policy == VFIO_PCI_TPH_POLICY_NO_ST)
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET,
+				 sizeof(tph_st));
+	if (ret != 1)
+		return ret;
+
+	if (copy_from_user(&tph_st, arg, sizeof(tph_st)))
+		return -EFAULT;
+
+	user_flags = tph_st.flags;
+	extended = !!(user_flags & VFIO_DEVICE_TPH_EXTENDED);
+	if (extended && !pcie_ext_tph_supported(vdev->pdev))
+		return -EOPNOTSUPP;
+	user_flags &= ~VFIO_DEVICE_TPH_EXTENDED;
+
+	stop_on_zero = !!(user_flags & VFIO_DEVICE_TPH_REQUIRE_ST);
+	if (stop_on_zero && (user_flags & VFIO_DEVICE_TPH_SRC_NONE))
+		return -EINVAL;
+	user_flags &= ~VFIO_DEVICE_TPH_REQUIRE_ST;
+
+	allow_src = vfio_pci_get_tph_st_allow_src(vdev);
+	if (user_flags & ~allow_src || !is_power_of_2(user_flags))
+		return -EINVAL;
+
+	st_size = vfio_pci_get_tph_st_size(vdev);
+	if (tph_st.start >= st_size || tph_st.count > st_size - tph_st.start ||
+		tph_st.count == 0)
+		return -EINVAL;
+
+	if (!(user_flags & VFIO_DEVICE_TPH_SRC_NONE)) {
+		src_hndl = memdup_array_user(u64_to_user_ptr(tph_st.dests),
+					     tph_st.count, sizeof(u32));
+		if (IS_ERR(src_hndl))
+			return PTR_ERR(src_hndl);
+	}
+	tags = kcalloc(tph_st.count, sizeof(u16), GFP_KERNEL_ACCOUNT);
+	if (!tags) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	for (i = 0; i < tph_st.count; i++) {
+		ret = vfio_pci_get_tph_st_tag(pdev, user_flags,
+					      src_hndl ? src_hndl[i] : 0,
+					      extended, &tags[i]);
+		if (ret || (stop_on_zero && tags[i] == 0))
+			break;
+	}
+
+	if (i > 0) {
+		cmd = vfio_pci_memory_lock_and_enable(vdev);
+		ret = pcie_tph_set_st_entries(pdev, tph_st.start, i, tags);
+		vfio_pci_memory_unlock_and_restore(vdev, cmd);
+	}
+
+out:
+	kfree(tags);
+	kfree(src_hndl);
+	return ret;
+}
+
 int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 				void __user *arg, size_t argsz)
 {
@@ -1754,6 +1901,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 	case VFIO_DEVICE_FEATURE_TPH_RESOLVE:
 		return vfio_pci_core_feature_tph_resolve(vdev, flags,
 							 arg, argsz);
+	case VFIO_DEVICE_FEATURE_TPH_ST:
+		return vfio_pci_core_feature_tph_st(vdev, flags, arg, argsz);
 	default:
 		return -ENOTTY;
 	}
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index 7fbf938f69e1..7b89aab7c4c1 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -13,6 +13,7 @@
 
 enum vfio_pci_tph_policy {
 	VFIO_PCI_TPH_POLICY_NO_ST = 0,
+	VFIO_PCI_TPH_POLICY_IV_ST = 1,
 };
 
 struct vfio_pci_ioeventfd {
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index ae7a2f083cd5..4b2132c2794e 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -1580,6 +1580,7 @@ struct vfio_device_feature_dma_buf_tph {
 /* PCIe TPH device feature definitions for VFIO_DEVICE_FEATURE ioctl */
 #define VFIO_DEVICE_FEATURE_TPH		14
 #define VFIO_DEVICE_FEATURE_TPH_RESOLVE	15
+#define VFIO_DEVICE_FEATURE_TPH_ST	16
 
 /*
  * VFIO_DEVICE_FEATURE_TPH - Control and query PCI TPH capabilities
@@ -1590,8 +1591,8 @@ struct vfio_device_feature_dma_buf_tph {
  *
  * Userspace must first invoke SET on this feature to enable TPH support,
  * receive valid capability bits via GET, gain permission for using
- * VFIO_DEVICE_FEATURE_TPH_RESOLVE, and access the virtualized TPH
- * capability registers via VFIO config space accesses.
+ * VFIO_DEVICE_FEATURE_TPH_RESOLVE / VFIO_DEVICE_FEATURE_TPH_ST, and access
+ * the virtualized TPH capability registers via VFIO config space accesses.
  */
 struct vfio_device_feature_tph {
 	__u32 flags;
@@ -1599,6 +1600,12 @@ struct vfio_device_feature_tph {
 
 /* DMABUF source resolve processing hint supported */
 #define VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH	(1u << 0)
+/* DMABUF source TPH_ST batch programming supported */
+#define VFIO_DEVICE_TPH_CAP_ST_DMABUF		(1u << 1)
+/* CPU source TPH_ST batch programming supported */
+#define VFIO_DEVICE_TPH_CAP_ST_CPU		(1u << 2)
+/* NONE source (means steering tag is zero) TPH_ST batch programming supported */
+#define VFIO_DEVICE_TPH_CAP_ST_NONE		(1u << 3)
 
 /*
  * VFIO_DEVICE_FEATURE_TPH_RESOLVE - Resolve TPH attributes from given source
@@ -1626,10 +1633,45 @@ struct vfio_device_feature_tph_resolve {
 	__u16 rsv;
 };
 
+/*
+ * VFIO_DEVICE_FEATURE_TPH_ST - Batch program architected steering tag
+ * table entries
+ * Return value: >=0 = number of successfully written entries;
+ *               <0  = negative errno
+ *
+ * SET only
+ *
+ * Additional constraint: Userspace must complete two prerequisite steps
+ * before invoking this feature:
+ * 1. Call VFIO_DEVICE_FEATURE_TPH with SET to enable vfio TPH permission;
+ * 2. Write the TPH control register to activate device hardware TPH logic.
+ *
+ * @flags: IN - source type selector (see VFIO_DEVICE_TPH_SRC_*) and
+ *         steering tag namespace selector (see VFIO_DEVICE_TPH_EXTENDED) and
+ *         failure policy (see VFIO_DEVICE_TPH_REQUIRE_ST)
+ * @start: IN - first steering tag table index / MSI-X vector
+ * @count: IN - number of contiguous entries to program
+ * @dests: IN - user pointer to array of source u32 values
+ */
+struct vfio_device_feature_tph_st {
+	__u32 flags;
+	__u16 start;
+	__u16 count;
+	__aligned_u64 dests;
+};
+
 /* Source holds dma-buf fd */
 #define VFIO_DEVICE_TPH_SRC_DMABUF      (1u << 0)
 /* Use extended 16-bit steering tag namespace */
 #define VFIO_DEVICE_TPH_EXTENDED	(1u << 1)
+/* Source holds volatile memory CPU ID */
+#define VFIO_DEVICE_TPH_SRC_CPU_VOLATILE	(1u << 2)
+/* Source holds persistent memory CPU ID */
+#define VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT	(1u << 3)
+/* TPH_ST only: set steering tag to zero */
+#define VFIO_DEVICE_TPH_SRC_NONE	(1u << 4)
+/* TPH_ST only: fail if resolved steering tag is zero */
+#define VFIO_DEVICE_TPH_REQUIRE_ST	(1u << 5)
 
 /* -------- API for Type1 VFIO IOMMU -------- */
 
-- 
2.17.1

[PATCH v20 14/16] vfio/pci: Add DS-ST TPH policy and extend TPH_RESOLVE capabilities

[Chengwen Feng]

Further extend VFIO TPH policy model based on NO-ST/IV-ST baseline,
add DS-ST policy and enrich TPH_RESOLVE to support full ST resolution.

- Add VFIO_PCI_TPH_POLICY_DS_ST level 2 policy, extend module param range
  * NO-ST: only DMABUF PH resolve, no ST capability
  * IV-ST: DMABUF PH resolve, support NONE/DMABUF/CPU ST programming
  * DS-ST: support DMABUF & CPU resolve, full ST/PH reporting and ST
    programming
- Relax TPH control write restriction to allow NS/IV/DS mode selection
- Extend TPH_RESOLVE logic:
  * Support CPU volatile/persistent memory source resolution
  * DMABUF source returns both PH and ST under DS-ST policy
  * CPU source returns ST tag only
- Add new TPH capability bits for ST resolution: RESOLVE_DMABUF_ST,
  RESOLVE_CPU_ST
- Expose resolution capabilities conditionally based on DS policy and DSM
  support

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci.c        |  4 +-
 drivers/vfio/pci/vfio_pci_config.c |  2 +-
 drivers/vfio/pci/vfio_pci_core.c   | 67 +++++++++++++++++++++---------
 drivers/vfio/pci/vfio_pci_priv.h   |  1 +
 include/uapi/linux/vfio.h          |  9 +++-
 5 files changed, 60 insertions(+), 23 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 02c5c8283099..c407c217b649 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -64,7 +64,7 @@ static unsigned int tph_policy;
 static int tph_policy_set(const char *val, const struct kernel_param *kp)
 {
 	return param_set_uint_minmax(val, kp, VFIO_PCI_TPH_POLICY_NO_ST,
-				     VFIO_PCI_TPH_POLICY_IV_ST);
+				     VFIO_PCI_TPH_POLICY_DS_ST);
 }
 static const struct kernel_param_ops tph_param_ops = {
 	.set = tph_policy_set,
@@ -76,6 +76,8 @@ MODULE_PARM_DESC(tph_policy,
 	"0 = No-ST mode: RESOLVE only returns PH for DMABUF source;\n"
 	"                ST programming unavailable\n"
 	"1 = Add IV mode: RESOLVE only returns PH for DMABUF source;\n"
+	"                 ST program supports NONE/DMABUF/CPU sources\n"
+	"2 = Add DS mode: RESOLVE returns PH for DMABUF source, ST for DMABUF/CPU sources;\n"
 	"                 ST program supports NONE/DMABUF/CPU sources\n");
 
 static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev)
diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 8c68504839b9..97dad6db69e3 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1201,7 +1201,7 @@ static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
 	cap = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CAP]);
 	mode = FIELD_GET(PCI_TPH_CTRL_MODE_SEL_MASK, new_ctrl);
 	req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, new_ctrl);
-	if (mode > PCI_TPH_ST_IV_MODE || !(cap & (1u << mode)) || req == 0x2 ||
+	if (mode > PCI_TPH_ST_DS_MODE || !(cap & (1u << mode)) || req == 0x2 ||
 		(req == PCI_TPH_REQ_EXT_TPH && !(cap & PCI_TPH_CAP_EXT_TPH)))
 		goto restore; /* Drop invalid or unsupported write value */
 
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index df31f4e8766f..ae2e0e2252a1 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1611,6 +1611,24 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
 	return 0;
 }
 
+static u32 vfio_pci_get_tph_st_size(struct vfio_pci_core_device *vdev)
+{
+	struct pci_dev *pdev = vdev->pdev;
+	u32 loc = pcie_tph_get_st_table_loc(pdev);
+	int ret;
+
+	if (loc == PCI_TPH_LOC_CAP) {
+		return pcie_tph_get_st_table_size(pdev);
+	} else if (loc == PCI_TPH_LOC_MSIX) {
+		ret = pci_msix_vec_count(pdev);
+		if (ret < 0)
+			return 0;
+		return ret;
+	} else {
+		return 0;
+	}
+}
+
 static u32 vfio_pci_get_tph_capability(struct vfio_pci_core_device *vdev)
 {
 	u32 flags = VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH;
@@ -1622,6 +1640,17 @@ static u32 vfio_pci_get_tph_capability(struct vfio_pci_core_device *vdev)
 		if (pcie_tph_dsm_supported(vdev->pdev))
 			flags |= VFIO_DEVICE_TPH_CAP_ST_CPU;
 	}
+	if (vdev->tph_policy >= VFIO_PCI_TPH_POLICY_DS_ST &&
+		(vdev->tph_cap_virt & PCI_TPH_CAP_ST_DS)) {
+		if (vfio_pci_get_tph_st_size(vdev) > 0) {
+			flags |= VFIO_DEVICE_TPH_CAP_ST_NONE;
+			flags |= VFIO_DEVICE_TPH_CAP_ST_DMABUF;
+			if (pcie_tph_dsm_supported(vdev->pdev))
+				flags |= VFIO_DEVICE_TPH_CAP_ST_CPU;
+		}
+		if (pcie_tph_dsm_supported(vdev->pdev))
+			flags |= VFIO_DEVICE_TPH_CAP_RESOLVE_CPU_ST;
+	}
 
 	return flags;
 }
@@ -1666,8 +1695,12 @@ static u32 vfio_pci_get_tph_resolve_allow_src(struct vfio_pci_core_device *vdev)
 	u32 flags = vfio_pci_get_tph_capability(vdev);
 	u32 allow_src = 0;
 
-	if (flags & VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH)
+	if (flags & (VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH |
+		     VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_ST))
 		allow_src |= VFIO_DEVICE_TPH_SRC_DMABUF;
+	if (flags & VFIO_DEVICE_TPH_CAP_RESOLVE_CPU_ST)
+		allow_src |= VFIO_DEVICE_TPH_SRC_CPU_VOLATILE |
+			     VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT;
 
 	return allow_src;
 }
@@ -1751,6 +1784,7 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
 
 	resolve.valid = 0;
 	resolve.ph = 0;
+	resolve.st = 0;
 
 	if (user_flags & VFIO_DEVICE_TPH_SRC_DMABUF) {
 		ret = vfio_pci_get_dmabuf_tph(resolve.src, extended,
@@ -1759,8 +1793,21 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
 			return ret;
 		resolve.ph = ph;
 		resolve.valid = VFIO_DEVICE_TPH_VALID_PH;
+		if (vdev->tph_policy >= VFIO_PCI_TPH_POLICY_DS_ST) {
+			resolve.st = tag;
+			resolve.valid |= VFIO_DEVICE_TPH_VALID_ST;
+		}
+		goto out;
 	}
 
+	ret = vfio_pci_get_tph_st_tag(vdev->pdev, user_flags, resolve.src,
+				      extended, &tag);
+	if (ret)
+		return ret;
+	resolve.valid |= VFIO_DEVICE_TPH_VALID_ST;
+	resolve.st = tag;
+
+out:
 	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
 }
 
@@ -1780,24 +1827,6 @@ static u32 vfio_pci_get_tph_st_allow_src(struct vfio_pci_core_device *vdev)
 	return allow_src;
 }
 
-static u32 vfio_pci_get_tph_st_size(struct vfio_pci_core_device *vdev)
-{
-	struct pci_dev *pdev = vdev->pdev;
-	u32 loc = pcie_tph_get_st_table_loc(pdev);
-	int ret;
-
-	if (loc == PCI_TPH_LOC_CAP) {
-		return pcie_tph_get_st_table_size(pdev);
-	} else if (loc == PCI_TPH_LOC_MSIX) {
-		ret = pci_msix_vec_count(pdev);
-		if (ret < 0)
-			return 0;
-		return ret;
-	} else {
-		return 0;
-	}
-}
-
 static int vfio_pci_core_feature_tph_st(struct vfio_pci_core_device *vdev,
 				u32 flags,
 				struct vfio_device_feature_tph_st __user *arg,
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index 7b89aab7c4c1..03c4a0414264 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -14,6 +14,7 @@
 enum vfio_pci_tph_policy {
 	VFIO_PCI_TPH_POLICY_NO_ST = 0,
 	VFIO_PCI_TPH_POLICY_IV_ST = 1,
+	VFIO_PCI_TPH_POLICY_DS_ST = 2,
 };
 
 struct vfio_pci_ioeventfd {
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 4b2132c2794e..9bc3cef7a89a 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -1606,6 +1606,10 @@ struct vfio_device_feature_tph {
 #define VFIO_DEVICE_TPH_CAP_ST_CPU		(1u << 2)
 /* NONE source (means steering tag is zero) TPH_ST batch programming supported */
 #define VFIO_DEVICE_TPH_CAP_ST_NONE		(1u << 3)
+/* DMABUF source resolve steering tag supported */
+#define VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_ST	(1u << 4)
+/* CPU source resolve steering tag supported */
+#define VFIO_DEVICE_TPH_CAP_RESOLVE_CPU_ST	(1u << 5)
 
 /*
  * VFIO_DEVICE_FEATURE_TPH_RESOLVE - Resolve TPH attributes from given source
@@ -1622,15 +1626,16 @@ struct vfio_device_feature_tph {
  * @valid: OUT - bitmap indicating valid output fields, see
  *         VFIO_DEVICE_TPH_VALID_*
  * @ph:    OUT - TPH processing hint (valid when VALID_PH is set in valid)
- * @rsv:   Must be zero, reserved for future extensions
+ * @st:    OUT - TPH steering tag (valid when VALID_ST is set in valid)
  */
 struct vfio_device_feature_tph_resolve {
 	__u32 flags;
 	__u32 src;
 	__u8  valid;
 #define VFIO_DEVICE_TPH_VALID_PH (1u << 0) /* ph holds valid processing hint */
+#define VFIO_DEVICE_TPH_VALID_ST (1u << 1) /* st holds valid steering tag */
 	__u8  ph;
-	__u16 rsv;
+	__u16 st;
 };
 
 /*
-- 
2.17.1

[PATCH v20 15/16] vfio/pci: Sync core PCI TPH software state across device lifecycle and resets

[Chengwen Feng]

Hardware resets clear PCI TPH registers, but core PCI driver software TPH
state can become stale across device open/close and reset paths.

- Call pcie_disable_tph() in vfio_pci_core_enable() to reset TPH state
  for new user sessions
- Call pcie_disable_tph() in vfio_pci_core_disable() to revert TPH state
  when releasing the device
- Call pcie_disable_tph() after successful VFIO_DEVICE_RESET and
  VFIO_DEVICE_PCI_HOT_RESET ioctls to sync software state with reset
  hardware

Ensure core PCI TPH software state always matches actual hardware state,
prevent residual TPH state leakage between user sessions and after resets.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index ae2e0e2252a1..c509adb76955 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -614,6 +614,8 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 
 	vdev->reset_works = !ret;
 	vdev->tph_opt_in = false;
+	/* Reset TPH status on new user session */
+	pcie_disable_tph(vdev->pdev);
 	pci_save_state(pdev);
 	vdev->pci_saved_state = pci_store_saved_state(pdev);
 	if (!vdev->pci_saved_state)
@@ -761,6 +763,9 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 
 	vfio_pci_zdev_close_device(vdev);
 
+	/* Reset TPH status on exit user session */
+	pcie_disable_tph(vdev->pdev);
+
 	/*
 	 * If we have saved state, restore it.  If we can reset the device,
 	 * even better.  Resetting with current state seems better than
@@ -1562,6 +1567,7 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
 	struct vfio_pci_core_device *vdev =
 		container_of(core_vdev, struct vfio_pci_core_device, vdev);
 	void __user *uarg = (void __user *)arg;
+	int ret;
 
 	switch (cmd) {
 	case VFIO_DEVICE_GET_INFO:
@@ -1573,9 +1579,15 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
 	case VFIO_DEVICE_IOEVENTFD:
 		return vfio_pci_ioctl_ioeventfd(vdev, uarg);
 	case VFIO_DEVICE_PCI_HOT_RESET:
-		return vfio_pci_ioctl_pci_hot_reset(vdev, uarg);
+		ret = vfio_pci_ioctl_pci_hot_reset(vdev, uarg);
+		if (!ret)
+			pcie_disable_tph(vdev->pdev);
+		return ret;
 	case VFIO_DEVICE_RESET:
-		return vfio_pci_ioctl_reset(vdev, uarg);
+		ret = vfio_pci_ioctl_reset(vdev, uarg);
+		if (!ret)
+			pcie_disable_tph(vdev->pdev);
+		return ret;
 	case VFIO_DEVICE_SET_IRQS:
 		return vfio_pci_ioctl_set_irqs(vdev, uarg);
 	default:
-- 
2.17.1

[PATCH v20 16/16] vfio/pci: Expose per-device TPH state via debugfs

[Chengwen Feng]

Add read-only debugfs file "tph" to report per-device TPH runtime state:
- effective tph_policy mode
- virtual TPH capability value
- tph_opt-in user enable state

Align debugfs comments to include tph_policy and new tph debug entry.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index c509adb76955..b8d9d840a93b 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -130,10 +130,20 @@ static int vfio_pci_core_debugfs_disable_idle_d3(struct seq_file *seq,
 	return 0;
 }
 
+static int vfio_pci_core_debugfs_tph(struct seq_file *seq,
+					    void *data)
+{
+	struct vfio_pci_core_device *vdev = vfio_pci_core_debugfs_private(seq);
+
+	seq_printf(seq, "policy=%u cap-virt=0x%x opt-in=%u\n",
+		vdev->tph_policy, vdev->tph_cap_virt, vdev->tph_opt_in);
+	return 0;
+}
+
 /*
- * disable_idle_d3 and nointxmask are writable module parameters latched
- * per device at init, so a device's effective value can differ from the
- * current parameter setting.  Expose the per-device (read-only) values
+ * disable_idle_d3, nointxmask and tph_policy are writable module parameters
+ * latched per device at init, so a device's effective value can differ from
+ * the current parameter setting.  Expose the per-device (read-only) values
  * here for visibility; read-only parameters can't drift and are omitted.
  */
 static void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
@@ -149,6 +159,8 @@ static void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
 				    vfio_pci_core_debugfs_nointxmask);
 	debugfs_create_devm_seqfile(dev, "disable_idle_d3", pci_dir,
 				    vfio_pci_core_debugfs_disable_idle_d3);
+	debugfs_create_devm_seqfile(dev, "tph", pci_dir,
+				    vfio_pci_core_debugfs_tph);
 }
 #else
 static inline void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
-- 
2.17.1

Re: [PATCH v20 02/16] dma-buf: add optional get_pci_tph() callback

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The kernel-doc comment references a non-existent callback `move_notify` in `struct dma_buf_attach_ops`.
--

--- Patch [2]: [PATCH v20 02/16] dma-buf: add optional get_pci_tph() callback ---
commit 799bc3a75b6c6b1dc555780b54878e41e3161d8b
Author: Zhiping Zhang <zhipingz@meta.com>

dma-buf: add optional get_pci_tph() callback

This commit adds an optional get_pci_tph callback to dma_buf_ops and a
corresponding wrapper function dma_buf_get_pci_tph() for importers. It
allows importers to query the exporter for PCIe TLP Processing Hint
namespaces and matching ST/PH tuples.

> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index d504c636dc294..e8314ed240d33 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
[ ... ]
> @@ -1144,6 +1146,36 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
>  }
>  EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, "DMA_BUF");
>  
> +/**
> + * dma_buf_get_pci_tph - Retrieve PCIe TLP Processing Hint (TPH) metadata
> + * @dmabuf: DMA buffer to query
> + * @extended: false for 8-bit ST, true for 16-bit Extended ST
> + * @steering_tag: returns the raw steering tag for the requested namespace
> + * @ph: returns the TPH processing hint
> + *
> + * Wrapper for the optional &dma_buf_ops.get_pci_tph callback.
> + *
> + * Must be called with &dma_buf.resv held. Returns -EOPNOTSUPP if the
> + * exporter does not implement the callback or has no metadata for the
> + * requested namespace.
> + *
> + * The returned steering tag and processing hint are only valid until the
> + * exporter invalidates the current mapping (signalled to the importer via
> + * &dma_buf_attach_ops.move_notify). Importers must re-query after a new

[Severity: Low]
This isn't a bug, but there is no move_notify callback in
struct dma_buf_attach_ops. Should this reference the invalidate_mappings
callback instead?

> + * mapping is established following invalidation.
> + */
> +int dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
> +			u16 *steering_tag, u8 *ph)

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=2

Re: [PATCH v20 03/16] vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature

[sashiko-bot]

> From: Zhiping Zhang <zhipingz@meta.com>
> 
> Implement dma-buf get_pci_tph for vfio-pci exported dma-bufs and add
> VFIO_DEVICE_FEATURE_DMA_BUF_TPH so userspace can publish TPH metadata
> for a VFIO-owned device.
> 
> 8-bit ST and 16-bit Extended ST are distinct PCIe TPH namespaces; the
> uAPI carries both with explicit validity flags, and get_pci_tph()
> returns the value matching the importer's requested namespace or
> -EOPNOTSUPP.
> 
> Publish and read the TPH descriptor under dmabuf->resv, matching the
> locking used for other importer-visible dma-buf state. The SET ioctl
> takes dma_resv_lock_interruptible(), while the callback runs under
> DMA-buf's asserted resv lock.
> [ ... ]
> PCI_EXP_DEVCAP2_TPH_COMP_EXT_TPH. Make PROBE follow the same hardware
> gate so the feature only probes as supported when the device can really
> consume it.
> 
> Signed-off-by: Zhiping Zhang <zhipingz@meta.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=3

Re: [PATCH v20 01/16] PCI/TPH: Add requester/completer type helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The internal get_rp_completer_type() function fails to fold the reserved 0b10 encoding into NONE, which can lead to writing a reserved value to the device's TPH Control Register and causes pcie_tph_enabled_req_type() to return an undocumented reserved value.
--

--- Patch [1]: [PATCH v20 01/16] PCI/TPH: Add requester/completer type helpers ---
commit f0ced626715c0e05a18f4a1a24c4e5840b72c4a0
Author: Zhiping Zhang <zhipingz@meta.com>

    PCI/TPH: Add requester/completer type helpers
    
    Add pcie_tph_enabled_req_type() so drivers can query the enabled TPH
    requester mode without reaching into pci_dev internals.
    
    Add pcie_tph_completer_type() so drivers that publish TPH metadata for
    a device acting as a completer can gate on the "TPH Completer
    Supported" field of Device Capabilities 2 (bits 13:12,
    PCI_EXP_DEVCAP2_TPH_COMP_MASK) rather than reusing requester-side
    state. Fold the reserved 0b10 encoding into NONE so callers only see
    the defined values.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index 655ffd60e62f6..e7693fd9d6763 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c
> @@ -173,6 +173,51 @@ u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
>  }
>  EXPORT_SYMBOL(pcie_tph_get_st_table_loc);
>  
> +/**
> + * pcie_tph_enabled_req_type - Return the device's enabled TPH requester type
> + * @pdev: PCI device to query
> + *
> + * Return: PCI_TPH_REQ_DISABLE, PCI_TPH_REQ_TPH_ONLY or PCI_TPH_REQ_EXT_TPH.
> + */
> +u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
> +{
> +	return pdev->tph_req_type;
> +}
> +EXPORT_SYMBOL(pcie_tph_enabled_req_type);

[Severity: High]
This is a pre-existing issue, but does pdev->tph_req_type correctly avoid
returning the reserved 0b10 encoding?

While pcie_tph_completer_type() added in this patch explicitly folds the
reserved 0b10 encoding into NONE, the pre-existing internal
get_rp_completer_type() function in drivers/pci/tph.c directly returns
the field value without folding the reserved state:

drivers/pci/tph.c:get_rp_completer_type() {
    ...
    return FIELD_GET(PCI_EXP_DEVCAP2_TPH_COMP_MASK, reg);
}

If a Root Port returns the reserved value 2 (0b10), pcie_enable_tph() handles
it like this:

drivers/pci/tph.c:pcie_enable_tph() {
    ...
    if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) {
        rp_req_type = get_rp_completer_type(pdev);

        /* Final req_type is the smallest value of two */
        pdev->tph_req_type = min(pdev->tph_req_type, rp_req_type);
    }
    ...
    FIELD_MODIFY(PCI_TPH_CTRL_REQ_EN_MASK, &reg, pdev->tph_req_type);
}

If the endpoint supports PCI_TPH_REQ_EXT_TPH (3), pdev->tph_req_type becomes
min(3, 2) = 2.

This bypassed value 2 is then written directly to the endpoint's TPH Requester
Enable field, which violates the PCIe specification and could cause undefined
hardware behavior. Furthermore, it causes this new pcie_tph_enabled_req_type()
function to leak this undocumented reserved value to external callers, which
contradicts the function documentation.

Should get_rp_completer_type() be updated to handle the reserved 0b10 encoding
in the same way as pcie_tph_completer_type()?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=1

Re: [PATCH v20 04/16] PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction

[sashiko-bot]

> pcie_tph_get_st_table_loc() incorrectly uses FIELD_GET(), which shifts the
> field value to bit 0. But the function is designed to return raw
> PCI_TPH_LOC_* values as defined in the function comment.
> 
> This causes incorrect ST table location detection. Fix it by using bitwise
> AND with PCI_TPH_CAP_LOC_MASK to return the unshifted field value matching
> the function specification.
> 
> This doesn't make a difference to mlx5_st_create(), the lone external
> caller, because it only checks for PCI_TPH_LOC_NONE (0), but will be needed
> for callers that check for PCI_TPH_LOC_CAP or PCI_TPH_LOC_MSIX.
> 
> Also add tph_cap validation for pcie_tph_get_st_table_loc() to prevent
> invalid PCI configuration space access when TPH is not supported. Add stub
> functions for pcie_tph_get_st_table_size() and pcie_tph_get_st_table_loc()
> when !CONFIG_PCIE_TPH.
> 
> Fixes: d2e8a34876ce ("PCI/TPH: Add Steering Tag support")
> Cc: stable@vger.kernel.org
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=4

Re: [PATCH v20 06/16] PCI/TPH: Cache negotiated max requester type at probe time

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Medium] get_rp_completer_type() does not validate the Root Port's TPH Completer Supported field, allowing a reserved value (2) to be propagated to the endpoint's TPH Requester Enable control register.
--

--- Patch [6]: [PATCH v20 06/16] PCI/TPH: Cache negotiated max requester type at probe time ---
commit 29ae72049d4aa847803163ffb3618441007a362d
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Cache negotiated max requester type at probe time

pcie_enable_tph() recalculates the TPH requester capability and queries
root port completer type on every invocation, causing redundant config
space reads. Calculate the negotiated maximum requester type once during
pci_tph_init() and cache it in a new pdev->tph_max_type bitfield. Simplify
the enable path by reusing the cached value to eliminate repeated hardware
access.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index 41084a45b0bbc..d123f91775c65 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c
[ ... ]
> @@ -551,13 +536,31 @@ void pci_no_tph(void)
>  
>  void pci_tph_init(struct pci_dev *pdev)
>  {
> +	u8 tph_req_type, rp_req_type;
>  	int num_entries;
>  	u32 save_size;
> +	u32 reg = 0;
>  
>  	pdev->tph_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_TPH);
>  	if (!pdev->tph_cap)
>  		return;
>  
> +	/* Get req_type supported by device and its Root Port */
> +	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
> +	if (FIELD_GET(PCI_TPH_CAP_EXT_TPH, reg))
> +		tph_req_type = PCI_TPH_REQ_EXT_TPH;
> +	else
> +		tph_req_type = PCI_TPH_REQ_TPH_ONLY;
> +
> +	/* Check if the device is behind a Root Port */
> +	if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) {
> +		rp_req_type = get_rp_completer_type(pdev);
> +		/* Final req_type is the smallest value of two */
> +		tph_req_type = min(tph_req_type, rp_req_type);
> +	}
> +
> +	pdev->tph_max_type = tph_req_type;

[Severity: Medium]
This isn't a bug introduced by this patch, but this caching logic embeds a
pre-existing issue where get_rp_completer_type() does not validate the Root
Port's TPH Completer Supported field.

If a non-compliant Root Port returns the reserved value 10b (2), could this
unvalidated value be propagated into pdev->tph_max_type and later written
to the Endpoint's TPH Requester Enable control register by
pcie_enable_tph()?

Would writing this reserved value to the endpoint's TPH control register
cause unpredictable hardware behavior or PCIe bus errors?

>  
>  	num_entries = pcie_tph_get_st_table_size(pdev);
>  	save_size = sizeof(u32) + num_entries * sizeof(u16);
>  	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=6

Re: [PATCH v20 07/16] PCI/TPH: Refactor pcie_enable_tph and add requester variant

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The documentation in `Documentation/PCI/tph.rst` was not updated to reflect the signature change of `pcie_enable_tph()` or the addition of the newly exported `pcie_enable_tph_ext()` API.
--

--- Patch [7]: [PATCH v20 07/16] PCI/TPH: Refactor pcie_enable_tph and add requester variant ---
commit 644907fcc1b63e61928d9441e443b763ecf84aa7
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Refactor pcie_enable_tph and add requester variant

This commit refactors the TPH enable logic by renaming the original function
to pcie_enable_tph_ext() and accepting an explicit requester type argument.
It also updates the signature of the pcie_enable_tph() compatibility wrapper
to use u8 for the mode parameter.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index d123f91775c65..1239a919b9a49 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c

[ ... ]

> @@ -463,6 +465,23 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
>  
>  	return 0;
>  }
> +EXPORT_SYMBOL(pcie_enable_tph_ext);
> +
> +/**
> + * pcie_enable_tph - Enable TPH support for device using a specific ST mode
> + * @pdev: PCI device
> + * @mode: ST mode to enable: PCI_TPH_ST_(NS_MODE / IV_MODE / DS_MODE)
> + *
> + * Wrapper for pcie_enable_tph_ext(), automatically uses the pre-cached
> + * tph_max_type value (negotiated maximum requester type between device and
> + * root port).
> + *
> + * Return: 0 on success, otherwise negative value (-errno)
> + */
> +int pcie_enable_tph(struct pci_dev *pdev, u8 mode)

[Severity: Low]
Should Documentation/PCI/tph.rst be updated to reflect these changes?

The documentation currently shows the signature as:

  int pcie_enable_tph(struct pci_dev *pdev, int mode);

It also does not mention the newly exported pcie_enable_tph_ext() API that
driver authors might need to use.

> +{
> +	return pcie_enable_tph_ext(pdev, mode, pdev->tph_max_type);
> +}
>  EXPORT_SYMBOL(pcie_enable_tph);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=7

Re: [PATCH v20 08/16] PCI/TPH: Refactor pcie_tph_get_cpu_st & add requester variant

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing length check for ACPI buffer in tph_invoke_dsm() leads to a heap out-of-bounds read.
--

--- Patch [8]: [PATCH v20 08/16] PCI/TPH: Refactor pcie_tph_get_cpu_st & add requester variant ---
commit a4213c1c2a9777e1dab404b096c528682a773636
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Refactor pcie_tph_get_cpu_st & add requester variant

This commit refactors pcie_tph_get_cpu_st() into a compatibility wrapper
and introduces pcie_tph_get_cpu_st_ext() to allow explicit TPH requester
type selection. It also adds validation for the requester type against
hardware negotiated capabilities.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c

[ ... ]

> @@ -313,17 +319,38 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
>  		return -EINVAL;
>  	}
>  
> -	*tag = tph_extract_tag(mem_type, pdev->tph_req_type, &info);
> +	*tag = tph_extract_tag(mem_type, req_type, &info);

[Severity: High]
This is a pre-existing issue, but while reviewing this code I noticed a
potential out-of-bounds read in tph_invoke_dsm() when it populates the
info union.

When tph_invoke_dsm() evaluates the ACPI _DSM method, it verifies the return
type is a buffer, but it doesn't verify the buffer length before casting the
pointer to a 64-bit integer and dereferencing it:

drivers/pci/tph.c:tph_invoke_dsm() {
    ...
    if (out_obj->type != ACPI_TYPE_BUFFER) {
        ACPI_FREE(out_obj);
        return AE_ERROR;
    }

    st_out->value = *((u64 *)(out_obj->buffer.pointer));
    ...
}

Could this cause a heap out-of-bounds read if buggy ACPI firmware provides a
buffer that is smaller than 8 bytes?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=8

Re: [PATCH v20 05/16] PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing

[sashiko-bot]

> Split tph_enabled from shared pci_dev bitfield into spare bit of tph_cap's
> u16: tph_cap is immutable post-enumeration (15 bits for offset), remaining
> 1 bit stores tph_enabled. Removes cross-bitfield concurrent write hazards
> highlighted by Sashiko after VFIO TPH exposure. No functional changes.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=5

Re: [PATCH v20 09/16] PCI/TPH: Add macros to check standard and extended TPH support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The macro `pcie_std_tph_supported()` erroneously returns true if `tph_max_type` is set to the reserved TPH requester type value 2.
- [Low] The helper macros `pcie_std_tph_supported` and `pcie_ext_tph_supported` are implemented as macros rather than static inline functions, causing the !CONFIG_PCIE_TPH stubs to silently drop the pdev argument.
--

--- Patch [9]: [PATCH v20 09/16] PCI/TPH: Add macros to check standard and extended TPH support ---
commit 62018f833ad520cae3d9509aa5138e780c6df62a
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Add macros to check standard and extended TPH support

This commit adds helper macros pcie_std_tph_supported() and
pcie_ext_tph_supported() for checking TPH capabilities. It relies on the
cached tph_max_type value and provides stub definitions for when
CONFIG_PCIE_TPH is disabled.

> diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
> index 6901f6e2f8c7a..04b521b7c2c7c 100644
> --- a/include/linux/pci-tph.h
> +++ b/include/linux/pci-tph.h

[ ... ]

> @@ -23,6 +23,18 @@ enum tph_mem_type {
>  };
>  
>  #ifdef CONFIG_PCIE_TPH
> +/**
> + * pcie_std_tph_supported - check standard TPH requester support
> + * @pdev: PCI device
> + */
> +#define pcie_std_tph_supported(pdev) \
> +	((pdev)->tph_max_type >= PCI_TPH_REQ_TPH_ONLY)

[Severity: Medium]
Will this macro falsely return true if the hardware advertises the reserved
value 2? 

If a PCIe Root Port incorrectly advertises the reserved value 10b (2) for its
TPH Completer Supported field, pci_tph_init() caches 2 into pdev->tph_max_type.
Since 2 >= PCI_TPH_REQ_TPH_ONLY (which is 1), this evaluates to true. However,
core functions like pcie_enable_tph_ext() explicitly reject req_type == 2 with
-EINVAL.

> +/**
> + * pcie_ext_tph_supported - check extended TPH requester support
> + * @pdev: PCI device
> + */
> +#define pcie_ext_tph_supported(pdev) \
> +	((pdev)->tph_max_type == PCI_TPH_REQ_EXT_TPH)

[ ... ]

> @@ -37,6 +49,8 @@ u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
>  u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
>  u8 pcie_tph_completer_type(struct pci_dev *pdev);
>  #else
> +#define pcie_std_tph_supported(pdev) false
> +#define pcie_ext_tph_supported(pdev) false

[Severity: Low]
Could these stub macros cause unused variable build warnings when
CONFIG_PCIE_TPH is disabled, since they silently drop the pdev argument?

Would it be better to implement these as static inline functions to guarantee
consistent type checking and evaluation semantics across all configurations?

>  static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
>  					unsigned int index, u16 tag)
>  { return -EINVAL; }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=9

Re: [PATCH v20 12/16] vfio/pci: Add basic NO-ST TPH VFIO device feature support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Unconditional release of the DMA reservation lock in `vfio_pci_get_dmabuf_tph()` when acquisition fails.
- [Medium] Missing validation for the reserved field `rsv` in `struct vfio_device_feature_tph_resolve`.
--

--- Patch [12]: [PATCH v20 12/16] vfio/pci: Add basic NO-ST TPH VFIO device feature support ---
commit e2b4dfe99fe4b36b6dce86a257a0fa1605e500b6
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Add basic NO-ST TPH VFIO device feature support

Implement basic PCIe TPH support in NO-ST mode, adding VFIO_DEVICE_FEATURE_TPH
and VFIO_DEVICE_FEATURE_TPH_RESOLVE to allow TPH operations after opt-in.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 4e5e34a77b76d..82582c8ef69da 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
[ ... ]
> @@ -1607,6 +1611,123 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
[ ... ]
> +static int vfio_pci_get_dmabuf_tph(int fd, bool extended, u16 *st, u8 *ph)
> +{
> +	struct dma_buf *dmabuf;
> +	int ret;
> +
> +	dmabuf = dma_buf_get(fd);
> +	if (IS_ERR(dmabuf))
> +		return PTR_ERR(dmabuf);
> +
> +	ret = dma_resv_lock_interruptible(dmabuf->resv, NULL);
> +	if (ret == 0)
> +		ret = dma_buf_get_pci_tph(dmabuf, extended, st, ph);
> +	dma_resv_unlock(dmabuf->resv);

[Severity: High]
Does this code unconditionally unlock the DMA reservation lock even if the
lock acquisition fails? 

If a signal interrupts dma_resv_lock_interruptible(), it will return -EINTR,
skipping the read but still calling dma_resv_unlock(). Can this underflow the
WW mutex state and lead to a panic?

> +
> +	dma_buf_put(dmabuf);
> +	return ret;
> +}
> +
> +static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
> +			u32 flags,
> +			struct vfio_device_feature_tph_resolve __user *arg,
> +			size_t argsz)
> +{
> +	struct vfio_device_feature_tph_resolve resolve;
> +	u32 user_flags, allow_src;
> +	bool extended;
> +	u16 tag = 0;
> +	u8 ph = 0;
> +	int ret;
> +
> +	if (!vdev->tph_opt_in)
> +		return -EOPNOTSUPP;
> +
> +	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET,
> +				 sizeof(resolve));
> +	if (ret != 1)
> +		return ret;
> +
> +	if (copy_from_user(&resolve, arg, sizeof(resolve)))
> +		return -EFAULT;
> +
> +	user_flags = resolve.flags;

[Severity: Medium]
Does this code need to validate that the reserved field resolve.rsv is zero?

Without checking it here, userspace could pass garbage data, which might
prevent safely extending this UAPI structure in the future without breaking
existing applications.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=12

Re: [PATCH v20 10/16] PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming helper

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

New issues:
- [High] API return value collision: PCIBIOS error codes masquerade as successful batch writes.
- [High] Partial success return value hides fatal global TPH disablement.
- [Medium] Calling the batch function with count=0 disrupts hardware TPH capability state.

Pre-existing issues:
- [High] Integer overflow in bounds check allows out-of-bounds config space writes.
- [Medium] Lockless data races on PCI_TPH_CTRL during concurrent updates.
--

--- Patch [10]: [PATCH v20 10/16] PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming helper ---
commit ef45067b72f46179b7e7c22243bc81d01093a5bd
Author: Chengwen Feng <fengchengwen@huawei.com>
Subject: PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming helper

This patch introduces a batch variant for programming contiguous Steering Tag
(ST) table entries. It modifies the existing pcie_tph_set_st_entry to act as a
wrapper around the new pcie_tph_set_st_entries helper. The batch helper
toggles TPH once per batch for better performance.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index 633dc6308b68c..8194f8b45e3c6 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c

[ ... ]

> @@ -378,7 +376,7 @@ int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
>  
>  	/* No need to write tag if device is in "No ST Mode" */
>  	if (pdev->tph_mode == PCI_TPH_ST_NS_MODE)
> -		return 0;
> +		return count;
>  
>  	/*
>  	 * Disable TPH before updating ST to avoid potential instability as
>  	 * cautioned in PCIe r6.2, sec 6.17.3, "ST Modes of Operation"
>  	 */

[Severity: Medium]
Will this unnecessarily disrupt the hardware state if the function is called
with count = 0? Since the loop will be bypassed, TPH will be disabled and
immediately re-enabled without writing any tags.

>  	set_ctrl_reg_req_en(pdev, PCI_TPH_REQ_DISABLE);

[Severity: Medium]
This is a pre-existing issue, but can concurrent updates cause lockless data
races on the PCI_TPH_CTRL register?

If multiple threads attempt to program the ST table concurrently, they may
race in set_ctrl_reg_req_en() which performs a lockless read-modify-write.
Could one thread re-enable TPH while another is midway through writing its
entries?

>  
>  	loc = pcie_tph_get_st_table_loc(pdev);
> -
> -	switch (loc) {
> -	case PCI_TPH_LOC_MSIX:
> -		err = pci_msix_write_tph_tag(pdev, index, tag);
> -		break;
> -	case PCI_TPH_LOC_CAP:
> -		err = write_tag_to_st_table(pdev, index, tag);
> -		break;
> -	default:
> -		err = -EINVAL;
> -	}
> -
> -	if (err) {
> -		pcie_disable_tph(pdev);
> -		return err;
> +	for (i = 0; i < count; i++) {
> +		switch (loc) {
> +		case PCI_TPH_LOC_MSIX:
> +			err = pci_msix_write_tph_tag(pdev, start + i, tags[i]);
> +			break;
> +		case PCI_TPH_LOC_CAP:
> +			err = write_tag_to_st_table(pdev, start + i, tags[i]);

[Severity: High]
This is a pre-existing issue, but does this bounds check in
write_tag_to_st_table() have an integer overflow vulnerability?

write_tag_to_st_table(struct pci_dev *pdev, int index, u16 tag) {
    ...
    if (index >= st_table_size)
        return -ENXIO;
    ...
}

Since write_tag_to_st_table() takes a signed int for the index, and we are
passing unsigned int start + i, could a large start value overflow to a
negative integer? A negative index would bypass the bounds check and cause an
out-of-bounds config space write.

> +			break;
> +		default:
> +			err = -EINVAL;
> +		}
> +		if (err != 0) {
> +			pci_err(pdev, "failed to set steering tag: %s table, index=%u, tags=%u, ret=%d\n",
> +				(loc == PCI_TPH_LOC_MSIX) ? "MSI-X" : "ST",
> +				start + i, tags[i], err);
> +			pcie_disable_tph(pdev);
> +			return i > 0 ? i : err;

[Severity: High]
If write_tag_to_st_table() fails on the first iteration where i == 0, could it
return a positive error code from pci_write_config_word() such as
PCIBIOS_BAD_REGISTER_NUMBER?

Because the return statement yields err when i == 0, a positive error code
would be interpreted by the caller of pcie_tph_set_st_entries() as a
successful batch write count.

[Severity: High]
If the loop fails on an entry after the first, this calls
pcie_disable_tph(pdev) which disables TPH for the entire device. However,
returning i yields a positive partial success count to the caller.

Does this leave the caller assuming the first i entries are functional when
TPH has actually been completely disabled?

> +		}
>  	}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=10

Re: [PATCH v20 14/16] vfio/pci: Add DS-ST TPH policy and extend TPH_RESOLVE capabilities

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [Medium] The newly introduced UAPI capability flag `VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_ST` is never exposed to userspace.
- [Medium] DMABUF steering tag resolution returns a steering tag based solely on policy, ignoring physical device support.

Pre-existing issues:
- [High] Unbalanced lock release in `vfio_pci_get_dmabuf_tph()` due to unconditionally calling `dma_resv_unlock()`.
--

--- Patch [14]: [PATCH v20 14/16] vfio/pci: Add DS-ST TPH policy and extend TPH_RESOLVE capabilities ---
commit e8e0f1c694e6597598ac910df3f902bedd7601ee
Author: Chengwen Feng <fengchengwen@huawei.com>
Subject: vfio/pci: Add DS-ST TPH policy and extend TPH_RESOLVE capabilities

Extends VFIO TPH policy to add DS-ST level 2 policy, relaxing write 
restrictions and extending TPH_RESOLVE logic. Adds support for CPU
memory sources and DMABUF sources returning both PH and ST under the
DS-ST policy.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index df31f4e8766fa..ae2e0e2252a1e 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c

[ ... ]

> @@ -1622,6 +1640,17 @@ static u32 vfio_pci_get_tph_capability(struct vfio_pci_core_device *vdev)
>  		if (pcie_tph_dsm_supported(vdev->pdev))
>  			flags |= VFIO_DEVICE_TPH_CAP_ST_CPU;
>  	}
> +	if (vdev->tph_policy >= VFIO_PCI_TPH_POLICY_DS_ST &&
> +		(vdev->tph_cap_virt & PCI_TPH_CAP_ST_DS)) {
> +		if (vfio_pci_get_tph_st_size(vdev) > 0) {
> +			flags |= VFIO_DEVICE_TPH_CAP_ST_NONE;
> +			flags |= VFIO_DEVICE_TPH_CAP_ST_DMABUF;
> +			if (pcie_tph_dsm_supported(vdev->pdev))
> +				flags |= VFIO_DEVICE_TPH_CAP_ST_CPU;
> +		}
> +		if (pcie_tph_dsm_supported(vdev->pdev))
> +			flags |= VFIO_DEVICE_TPH_CAP_RESOLVE_CPU_ST;
> +	}

[Severity: Medium]
The commit message and headers mention adding the capability
VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_ST. Should it be added to the flags
bitmask here alongside VFIO_DEVICE_TPH_CAP_RESOLVE_CPU_ST? 

If it's left out, userspace might not be able to discover that the feature
is actually supported.

>  
>  	return flags;
>  }

[ ... ]

> @@ -1751,6 +1784,7 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
>  
>  	resolve.valid = 0;
>  	resolve.ph = 0;
> +	resolve.st = 0;
>  
>  	if (user_flags & VFIO_DEVICE_TPH_SRC_DMABUF) {
>  		ret = vfio_pci_get_dmabuf_tph(resolve.src, extended,

[Severity: High]
This isn't a bug introduced by this patch, but looking at
vfio_pci_get_dmabuf_tph() which is called here:

vfio_pci_get_dmabuf_tph() {
        ret = dma_resv_lock_interruptible(dmabuf->resv, NULL);
        if (ret == 0)
                ret = dma_buf_get_pci_tph(dmabuf, extended, st, ph);
        dma_resv_unlock(dmabuf->resv);
        ...
}

If dma_resv_lock_interruptible() fails (for example, if it's interrupted
by a signal and returns -EINTR), will the unconditional call to
dma_resv_unlock() corrupt the underlying ww_mutex state?

> @@ -1759,8 +1793,21 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
>  			return ret;
>  		resolve.ph = ph;
>  		resolve.valid = VFIO_DEVICE_TPH_VALID_PH;
> +		if (vdev->tph_policy >= VFIO_PCI_TPH_POLICY_DS_ST) {
> +			resolve.st = tag;
> +			resolve.valid |= VFIO_DEVICE_TPH_VALID_ST;
> +		}
> +		goto out;
>  	}

[Severity: Medium]
Does this evaluation need to also check if the physical device supports the
DS-ST mode?

If the global module policy allows DS-ST but the device lacks the underlying
capability (vdev->tph_cap_virt & PCI_TPH_CAP_ST_DS), it seems this could
still return a valid steering tag to userspace for a tag it cannot actually
program.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=14

Re: [PATCH v20 13/16] vfio/pci: Add IV-ST TPH policy and VFIO TPH_ST batch programming support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Unconditional call to `dma_resv_unlock` even when `dma_resv_lock_interruptible` fails, leading to unbalanced unlock.
--

--- Patch [13]: [PATCH v20 13/16] vfio/pci: Add IV-ST TPH policy and VFIO TPH_ST batch programming support ---
commit d4cf54f638f6e3b2fed97fa978cb257c72da35af
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Add IV-ST TPH policy and VFIO TPH_ST batch programming support
    
This commit extends VFIO TPH implementation to support IV-ST mode based on
the existing NO-ST baseline. It adds the VFIO_DEVICE_FEATURE_TPH_ST feature
to enable batch ST table programming for NONE, DMABUF, and CPU sources.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 82582c8ef69da..df31f4e8766fa 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c

[ ... ]

> @@ -1680,6 +1690,32 @@ static int vfio_pci_get_dmabuf_tph(int fd, bool extended, u16 *st, u8 *ph)
>  	return ret;
>  }
>  
> +static int vfio_pci_get_tph_st_tag(struct pci_dev *pdev, u32 src_bits,
> +				   u32 src_hndl, bool extended, u16 *st)
> +{
> +	u8 req_type = extended ? PCI_TPH_REQ_EXT_TPH : PCI_TPH_REQ_TPH_ONLY;
> +	int ret = 0;
> +	u8 ph;
> +
> +	if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF)
> +		ret = vfio_pci_get_dmabuf_tph(src_hndl, extended, st, &ph);

[Severity: High]
This isn't a bug introduced by this patch, but looking at the implementation
of vfio_pci_get_dmabuf_tph() which is called here, is there an unbalanced
unlock if the lock acquisition is interrupted?

In drivers/vfio/pci/vfio_pci_core.c:vfio_pci_get_dmabuf_tph():

	ret = dma_resv_lock_interruptible(dmabuf->resv, NULL);
	if (ret == 0)
		ret = dma_buf_get_pci_tph(dmabuf, extended, st, ph);
	dma_resv_unlock(dmabuf->resv);

If a signal is delivered to the thread while it is attempting to acquire the
lock, dma_resv_lock_interruptible() will return a non-zero error. The
if (ret == 0) check correctly skips the inner function, but does this code
still unconditionally execute dma_resv_unlock() and corrupt the lock state?

> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_VOLATILE)
> +		ret = pcie_tph_get_cpu_st_ext(pdev, TPH_MEM_TYPE_VM,
> +					      req_type, src_hndl, st);
> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT)
> +		ret = pcie_tph_get_cpu_st_ext(pdev, TPH_MEM_TYPE_PM,
> +					      req_type, src_hndl, st);
> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_NONE)
> +		*st = 0;
> +	else
> +		ret = -EINVAL;
> +
> +	if (ret != 0)
> +		*st = 0;
> +
> +	return ret;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=13

Re: [PATCH v20 16/16] vfio/pci: Expose per-device TPH state via debugfs

[sashiko-bot]

> Add read-only debugfs file "tph" to report per-device TPH runtime state:
> - effective tph_policy mode
> - virtual TPH capability value
> - tph_opt-in user enable state
> 
> Align debugfs comments to include tph_policy and new tph debug entry.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=16

Re: [PATCH v20 11/16] PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The newly added `pcie_tph_dsm_supported()` helper incorrectly evaluates the ACPI _DSM on the Host Bridge instead of the Root Port.
- [Medium] The newly added `pcie_tph_dsm_supported()` helper does not support Root Complex Integrated Endpoints (RCiEPs).
--

--- Patch [11]: [PATCH v20 11/16] PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM ---
commit b4b5ff3a7f819f0fbddf034e29a0aa1c625cb893
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM

This commit adds a new helper pcie_tph_dsm_supported() to check whether the
root port of a given PCI device exposes the TPH ST ACPI _DSM method. The
logic is guarded by CONFIG_ACPI and returns false if ACPI is disabled.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index 8194f8b45e3c6..e6a850ab2b037 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c
[ ... ]
> +bool pcie_tph_dsm_supported(struct pci_dev *pdev)
> +{
> +#ifdef CONFIG_ACPI
> +	struct pci_dev *rp = pcie_find_root_port(pdev);
> +	acpi_handle handle;
> +
> +	if (!rp || !rp->bus || !rp->bus->bridge)
> +		return false;

[Severity: Medium]
Will this check fail for Root Complex Integrated Endpoints (RCiEPs)?

Since RCiEPs do not have an upstream Root Port, pcie_find_root_port() will
return NULL and this function will return false. Should this include a
fallback to check the associated Root Complex Event Collector (RCEC) for
the _DSM?

> +
> +	handle = ACPI_HANDLE(rp->bus->bridge);

[Severity: Medium]
Is this evaluating the ACPI _DSM on the correct device?

By using rp->bus->bridge, this fetches the ACPI handle of the Host Bridge
(the parent of the Root Port's upstream bus). Since the TPH _DSM is
typically placed on the Root Port itself per the PCI Firmware Specification,
should this use ACPI_HANDLE(&rp->dev) instead to avoid a false negative?

> +	if (!handle)
> +		return false;
[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=11

Re: [PATCH v20 15/16] vfio/pci: Sync core PCI TPH software state across device lifecycle and resets

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] VFIO_DEVICE_PCI_HOT_RESET fails to synchronize software TPH state for all devices in the affected reset group.
- [High] Unlocked calls to `pcie_disable_tph()` in VFIO ioctl reset paths cause TOCTOU data races with concurrent TPH configuration.
- [High] TPH virtual configuration space (`vconfig`) remains stale after reset ioctls, permanently breaking future TPH enablement.
- [High] pcie_disable_tph() accesses PCI configuration space without holding a runtime PM reference, which can race with device suspension.
--

--- Patch [15]: [PATCH v20 15/16] vfio/pci: Sync core PCI TPH software state across device lifecycle and resets ---
commit 0b983b3e1196f13bb26ca2c1d7517ec00ad08a77
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Sync core PCI TPH software state across device lifecycle and resets

Hardware resets clear PCI TPH registers, but core PCI driver software TPH
state can become stale across device open/close and reset paths. This patch
calls pcie_disable_tph() during device enable/disable and after reset ioctls
to synchronize the software tracking state with the hardware.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index ae2e0e2252a1e..c509adb76955d 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
[ ... ]
> @@ -1573,9 +1579,15 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
>  	case VFIO_DEVICE_IOEVENTFD:
>  		return vfio_pci_ioctl_ioeventfd(vdev, uarg);
>  	case VFIO_DEVICE_PCI_HOT_RESET:
> -		return vfio_pci_ioctl_pci_hot_reset(vdev, uarg);
> +		ret = vfio_pci_ioctl_pci_hot_reset(vdev, uarg);
> +		if (!ret)
> +			pcie_disable_tph(vdev->pdev);

[Severity: High]
Does this successfully clear the software state for all devices in the reset
group?

When userspace invokes VFIO_DEVICE_PCI_HOT_RESET, the hardware TPH state
is cleared for all devices in the reset group. However, only the targeted
vdev has pcie_disable_tph() called.

Will other devices in the reset group retain pdev->tph_enabled == 1,
causing pcie_enable_tph_ext() to fail with -EBUSY when userspace attempts
to re-enable TPH later?


[Severity: High]
Can pcie_disable_tph() race with device suspension here?

vfio_pci_ioctl_pci_hot_reset() acquires PM runtime references for all devices
but drops them via pm_runtime_put() before returning. If the device is
configured for runtime PM, it may immediately begin suspending.

Since pcie_disable_tph() performs an unlocked pci_write_config_dword(),
does it risk accessing the PCI configuration space of a suspended device,
potentially leading to PCIe Completion Timeouts or dropped writes?

> +		return ret;
>  	case VFIO_DEVICE_RESET:
> -		return vfio_pci_ioctl_reset(vdev, uarg);
> +		ret = vfio_pci_ioctl_reset(vdev, uarg);
> +		if (!ret)
> +			pcie_disable_tph(vdev->pdev);

[Severity: High]
Is there a data race with concurrent TPH configuration?

The reset handlers drop dev_set->lock and vdev->memory_lock before
returning, allowing pcie_disable_tph() to run locklessly.

If a concurrent userspace thread writes to the TPH virtual config space, it
takes memory_lock and calls pcie_enable_tph_ext(). Can these two threads
concurrently modify PCI_TPH_CTRL and pdev->tph_enabled, corrupting the
tracking state?


[Severity: High]
Does this leave the virtual configuration space (vconfig) stale?

While pcie_disable_tph() clears the hardware state and pdev->tph_enabled,
it doesn't update vdev->vconfig.

If the guest OS later rewrites the config space to re-enable TPH,
vfio_tph_config_write() will see:

    new_ctrl = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CTRL]);
    if (new_ctrl == org_ctrl)
        goto out;

Since vconfig still contains the enabled state from before the reset, won't
this treat the write as a no-op, leaving TPH permanently disabled in hardware?

> +		return ret;
>  	case VFIO_DEVICE_SET_IRQS:
>  		return vfio_pci_ioctl_set_irqs(vdev, uarg);
>  	default:

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260714104325.54444-1-fengchengwen@huawei.com?part=15