[PATCH v19 00/18] vfio/pci: Add PCIe TPH support

[PATCH v19 00/18] vfio/pci: Add PCIe TPH support

[Chengwen Feng]

This patchset enables full userspace configurable PCIe TPH support for          
VFIO, which brings performance benefits for userspace workloads such as         
DPDK and SPDK.                                                                  
                                                                                
Currently VFIO only exposes read-only TPH capability registers to               
userspace, while all write operations are silently discarded. This              
prevents userspace from enabling and configuring TPH, limiting performance      
optimization opportunities.                                                     
                                                                                
Per PCIe spec 7.5.3.15: TPH Completer support is applicable to Root Ports       
and Endpoints, allowing Steering Tags to target host CPUs or peer devices       
for P2P transactions.                                                           
                                                                                
TPH usage model can be divided into three fundamental parts:                    
1. Retrieve Steering Tag:                                                       
   - Tags targeting host CPUs are obtained via platform methods (ACPI _DSM)     
     wrapped in pcie_tph_get_cpu_st(). Userspace requires a generic             
     interface to query these CPU-associated ST values.                         
   - Tags targeting peer devices are managed by userspace drivers, and set
     to kernel by dmabuf ioctl which address by zhiping's patchset.              
2. Program Steering Tag table:                                                  
   - For devices with standard ST table structures (in capability space or      
     MSI-X table), userspace needs a unified interface to configure ST          
     entries.                                                                   
   - Devices without standard ST tables are handled by userspace itself.        
3. Toggle device TPH Requester enable/disable state.

The patchset builds a complete, secure, staged TPH feature stack with core
design objectives:
1. Refactor & fix upstream PCI/TPH core implementation to provide clean, 
   safe, extendable helper APIs for VFIO consumption;
2. Introduce configurable per-device TPH policy for staged feature 
   enablement, supporting progressive enabling of No-ST/IV/DS/LITERAL 
   modes;
3. Fully virtualize PCIe TPH config space: read-only capability header, 
   virtually writable TPH_CTRL and ST-table entries, silent discard 
   invalid writes;
4. Implement unified VFIO user feature uAPI (TPH / TPH_RESOLVE / TPH_ST)
   for TPH enablement, tag resolution and batch ST table programming;

The patchset is split into two logical parts: the first eight patches fix       
and refactor core PCI/TPH kernel code to export required helper interfaces,
the remaining eight patches implement corresponding VFIO TPH virtualization
and uAPI step by step.

Based on earlier RFC work by Wathsala Vithanage
Based on zhiping's patchset of add TPH support for p2p access

---
v19:
- Address all comments from Alex
  - Drop CPU-to-ST translation of PCI/TPH
  - Re-implement vfio part, please refer:
    https://lore.kernel.org/linux-pci/20260626092232.53ed3a7c@shazbot.org/
- Add expose tph_policy via debugfs
v18-resend:                                                                     
- Fix apply failed only                                                         
v18:                                                                            
- Address all comments of [08/12] commit from Alex                              
  - Add document in sysfs-bus-pci                                               
  - Place the new field at the root port                                        
  - Add new filed only when root port support TPH completer and _DSM            
    method                                                                      
  - Support random offset read, return zero if cpu is offline                   
  - Zero-initialize the buffer rather than memset and reserved = 0              
- Fix git am fail for [10/12] commit                                            
- Fix folloing Sashiko review comments of [11/12] commit:                       
  - Add __GFP_ACCOUNT in thp_st_shadow allocation                               
  - Move reset tph_permit in vfio_pci_tph_init/deinit                           
  - Refine feature PROBE for `TPH_ST_CONFIG`, make sure probe OK when           
    enable_unsafe_tph is set                                                    
  - Add commit-log about why TPH ST shadow table is not cleared when the        
    VFIO device is reset 

Chengwen Feng (17):
  PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction
  PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing
  PCI/TPH: Cache TPH requester capability at probe time
  PCI/TPH: Refactor pcie_enable_tph & add explicit requester variant
  PCI/TPH: Refactor pcie_tph_get_cpu_st & add explicit variant
  PCI/TPH: Add pcie_tph_supported() helper to check TPH capability
    attributes
  PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST
    _DSM
  vfio/pci: Hide TPH capability when TPH is unsupported
  vfio/pci: Introduce tph policy parameter for staged TPH feature
    enablement
  vfio/pci: Virtualize PCIe TPH capability registers
  vfio/pci: Add dmabuf TPH metadata storage and fd query helper
  vfio/pci: Introduce VFIO_DEVICE_FEATURE_TPH family uapi for PCI TPH
    control
  vfio/pci: Implement VFIO_DEVICE_FEATURE_TPH and valid TPH config write
    support
  vfio/pci: Implement TPH_RESOLVE feature for DMABUF and CPU source
    resolving
  vfio/pci: Implement TPH_ST feature for batch ST table programming
  vfio/pci: Reset hardware TPH state on device enable/disable
  vfio/pci: Expose tph_policy via debugfs

Zhiping Zhang (1):
  PCI/TPH: Expose the enabled TPH requester type

 drivers/pci/tph.c                  | 246 +++++++++++++++++++++-------
 drivers/vfio/pci/vfio_pci.c        |  25 +++
 drivers/vfio/pci/vfio_pci_config.c | 129 +++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c   | 252 ++++++++++++++++++++++++++++-
 drivers/vfio/pci/vfio_pci_dmabuf.c |  49 +++++-
 drivers/vfio/pci/vfio_pci_priv.h   |  13 ++
 include/linux/pci-tph.h            |  25 +++
 include/linux/pci.h                |   6 +-
 include/linux/vfio_pci_core.h      |   2 +
 include/uapi/linux/vfio.h          |  62 +++++++
 10 files changed, 742 insertions(+), 67 deletions(-)

-- 
2.17.1

[PATCH v19 01/18] PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction

[Chengwen Feng]

pcie_tph_get_st_table_loc() incorrectly uses FIELD_GET(), which shifts the
field value to bit 0. But the function is designed to return raw
PCI_TPH_LOC_* values as defined in the function comment.

This causes incorrect ST table location detection. Fix it by using bitwise
AND with PCI_TPH_CAP_LOC_MASK to return the unshifted field value matching
the function specification.

This doesn't make a difference to mlx5_st_create(), the lone external
caller, because it only checks for PCI_TPH_LOC_NONE (0), but will be needed
for callers that check for PCI_TPH_LOC_CAP or PCI_TPH_LOC_MSIX.

Also add tph_cap validation for pcie_tph_get_st_table_loc() to prevent
invalid PCI configuration space access when TPH is not supported. Add stub
functions for pcie_tph_get_st_table_size() and pcie_tph_get_st_table_loc()
when !CONFIG_PCIE_TPH.

Fixes: d2e8a34876ce ("PCI/TPH: Add Steering Tag support")
Cc: stable@vger.kernel.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Reviewed-by: Alex Williamson <alex.williamson@nvidia.com>
Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/tph.c       | 12 +++++-------
 include/linux/pci-tph.h |  5 +++++
 2 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 655ffd60e62f..489487158b99 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -165,11 +165,14 @@ static u8 get_st_modes(struct pci_dev *pdev)
  */
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
 {
-	u32 reg;
+	u32 reg = 0;
+
+	if (!pdev->tph_cap)
+		return PCI_TPH_LOC_NONE;
 
 	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
 
-	return FIELD_GET(PCI_TPH_CAP_LOC_MASK, reg);
+	return reg & PCI_TPH_CAP_LOC_MASK;
 }
 EXPORT_SYMBOL(pcie_tph_get_st_table_loc);
 
@@ -184,9 +187,6 @@ u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
 
 	/* Check ST table location first */
 	loc = pcie_tph_get_st_table_loc(pdev);
-
-	/* Convert loc to match with PCI_TPH_LOC_* defined in pci_regs.h */
-	loc = FIELD_PREP(PCI_TPH_CAP_LOC_MASK, loc);
 	if (loc != PCI_TPH_LOC_CAP)
 		return 0;
 
@@ -315,8 +315,6 @@ int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, u16 tag)
 	set_ctrl_reg_req_en(pdev, PCI_TPH_REQ_DISABLE);
 
 	loc = pcie_tph_get_st_table_loc(pdev);
-	/* Convert loc to match with PCI_TPH_LOC_* */
-	loc = FIELD_PREP(PCI_TPH_CAP_LOC_MASK, loc);
 
 	switch (loc) {
 	case PCI_TPH_LOC_MSIX:
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index be68cd17f2f8..6f02b020d7d7 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -8,6 +8,7 @@
  */
 #ifndef LINUX_PCI_TPH_H
 #define LINUX_PCI_TPH_H
+#include <linux/pci.h>
 
 /*
  * According to the ECN for PCI Firmware Spec, Steering Tag can be different
@@ -41,6 +42,10 @@ static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
 static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
 { return -EINVAL; }
+static inline u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
+{ return 0; }
+static inline u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
+{ return PCI_TPH_LOC_NONE; }
 #endif
 
 #endif /* LINUX_PCI_TPH_H */
-- 
2.17.1

[PATCH v19 02/18] PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing

[Chengwen Feng]

Split tph_enabled from shared pci_dev bitfield into spare bit of tph_cap's
u16: tph_cap is immutable post-enumeration (15 bits for offset), remaining
1 bit stores tph_enabled. Removes cross-bitfield concurrent write hazards
highlighted by Sashiko after VFIO TPH exposure. No functional changes.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 include/linux/pci.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/pci.h b/include/linux/pci.h
index 64b308b6e61c..ce520523b152 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -472,7 +472,6 @@ struct pci_dev {
 	unsigned int	ats_enabled:1;		/* Address Translation Svc */
 	unsigned int	pasid_enabled:1;	/* Process Address Space ID */
 	unsigned int	pri_enabled:1;		/* Page Request Interface */
-	unsigned int	tph_enabled:1;		/* TLP Processing Hints */
 	unsigned int	fm_enabled:1;		/* Flit Mode (segment captured) */
 	unsigned int	is_managed:1;		/* Managed via devres */
 	unsigned int	is_msi_managed:1;	/* MSI release via devres installed */
@@ -588,7 +587,8 @@ struct pci_dev {
 	u8 reset_methods[PCI_NUM_RESET_METHODS]; /* In priority order */
 
 #ifdef CONFIG_PCIE_TPH
-	u16		tph_cap;	/* TPH capability offset */
+	u16		tph_cap:15;	/* TPH capability offset */
+	u16		tph_enabled:1;	/* Whether TPH is enabled */
 	u8		tph_mode;	/* TPH mode */
 	u8		tph_req_type;	/* TPH requester type */
 #endif
-- 
2.17.1

[PATCH v19 03/18] PCI/TPH: Cache TPH requester capability at probe time

[Chengwen Feng]

Calculate the negotiated TPH requester type from device and root port
capabilities once in pci_tph_init().

Add tph_ext_support flag to cache whether the device is allowed to
issue Extended TPH requests after topology negotiation. If the final
requester type is disabled, clear TPH capability to prevent usage.

Simplify pcie_enable_tph() by using the cached requester capability
instead of recalculating every time.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c   | 43 +++++++++++++++++++++++++------------------
 include/linux/pci.h |  4 +++-
 2 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 489487158b99..4097787ea98a 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -383,7 +383,6 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 {
 	u32 reg;
 	u8 dev_modes;
-	u8 rp_req_type;
 
 	/* Honor "notph" kernel parameter */
 	if (pci_tph_disabled)
@@ -403,23 +402,8 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 
 	pdev->tph_mode = mode;
 
-	/* Get req_type supported by device and its Root Port */
-	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
-	if (FIELD_GET(PCI_TPH_CAP_EXT_TPH, reg))
-		pdev->tph_req_type = PCI_TPH_REQ_EXT_TPH;
-	else
-		pdev->tph_req_type = PCI_TPH_REQ_TPH_ONLY;
-
-	/* Check if the device is behind a Root Port */
-	if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) {
-		rp_req_type = get_rp_completer_type(pdev);
-
-		/* Final req_type is the smallest value of two */
-		pdev->tph_req_type = min(pdev->tph_req_type, rp_req_type);
-	}
-
-	if (pdev->tph_req_type == PCI_TPH_REQ_DISABLE)
-		return -EINVAL;
+	pdev->tph_req_type = pdev->tph_ext_support ? PCI_TPH_REQ_EXT_TPH :
+						     PCI_TPH_REQ_TPH_ONLY;
 
 	/* Write them into TPH control register */
 	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CTRL, &reg);
@@ -506,13 +490,36 @@ void pci_no_tph(void)
 
 void pci_tph_init(struct pci_dev *pdev)
 {
+	u8 tph_req_type, rp_req_type;
 	int num_entries;
 	u32 save_size;
+	u32 reg = 0;
 
 	pdev->tph_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_TPH);
 	if (!pdev->tph_cap)
 		return;
 
+	/* Get req_type supported by device and its Root Port */
+	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CAP, &reg);
+	if (FIELD_GET(PCI_TPH_CAP_EXT_TPH, reg))
+		tph_req_type = PCI_TPH_REQ_EXT_TPH;
+	else
+		tph_req_type = PCI_TPH_REQ_TPH_ONLY;
+
+	/* Check if the device is behind a Root Port */
+	if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) {
+		rp_req_type = get_rp_completer_type(pdev);
+		/* Final req_type is the smallest value of two */
+		tph_req_type = min(tph_req_type, rp_req_type);
+	}
+
+	if (tph_req_type == PCI_TPH_REQ_DISABLE) {
+		pdev->tph_cap = 0;
+		return;
+	}
+
+	pdev->tph_ext_support = !!(tph_req_type == PCI_TPH_REQ_EXT_TPH);
+
 	num_entries = pcie_tph_get_st_table_size(pdev);
 	save_size = sizeof(u32) + num_entries * sizeof(u16);
 	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index ce520523b152..a99eeea53c81 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -587,7 +587,9 @@ struct pci_dev {
 	u8 reset_methods[PCI_NUM_RESET_METHODS]; /* In priority order */
 
 #ifdef CONFIG_PCIE_TPH
-	u16		tph_cap:15;	/* TPH capability offset */
+	u16		tph_cap:14;	/* TPH capability offset */
+	u16		tph_ext_support:1; /* Indicate whether Extended TPH
+					    * requester is supported */
 	u16		tph_enabled:1;	/* Whether TPH is enabled */
 	u8		tph_mode;	/* TPH mode */
 	u8		tph_req_type;	/* TPH requester type */
-- 
2.17.1

[PATCH v19 04/18] PCI/TPH: Refactor pcie_enable_tph & add explicit requester variant

[Chengwen Feng]

Refactor pcie_enable_tph implementation: extract core logic into static
internal enable_tph() helper accepting explicit requester type.

- Preserve original pcie_enable_tph() unchanged as auto wrapper; it
  auto-selects EXT/standard TPH requester per device capability, existing
  bnxt/mlx5 callers require zero modification.
- Add exported pcie_enable_tph_explicit() with bool 'extended' parameter
  for explicit STD/EXT selection, used by upcoming VFIO TPH support.

Input validation for EXT_TPH availability is retained inside helper to
reject invalid explicit EXT request if hardware does not support extended
requester.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 68 +++++++++++++++++++++++++++++------------
 include/linux/pci-tph.h |  4 +++
 2 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 4097787ea98a..c22eb7f4b0bd 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -363,23 +363,7 @@ void pcie_disable_tph(struct pci_dev *pdev)
 }
 EXPORT_SYMBOL(pcie_disable_tph);
 
-/**
- * pcie_enable_tph - Enable TPH support for device using a specific ST mode
- * @pdev: PCI device
- * @mode: ST mode to enable. Current supported modes include:
- *
- *   - PCI_TPH_ST_NS_MODE: NO ST Mode
- *   - PCI_TPH_ST_IV_MODE: Interrupt Vector Mode
- *   - PCI_TPH_ST_DS_MODE: Device Specific Mode
- *
- * Check whether the mode is actually supported by the device before enabling
- * and return an error if not. Additionally determine what types of requests,
- * TPH or extended TPH, can be issued by the device based on its TPH requester
- * capability and the Root Port's completer capability.
- *
- * Return: 0 on success, otherwise negative value (-errno)
- */
-int pcie_enable_tph(struct pci_dev *pdev, int mode)
+static int enable_tph(struct pci_dev *pdev, int mode, u8 req_type)
 {
 	u32 reg;
 	u8 dev_modes;
@@ -400,10 +384,11 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 	if (!((1 << mode) & dev_modes))
 		return -EINVAL;
 
-	pdev->tph_mode = mode;
+	if (req_type == PCI_TPH_REQ_EXT_TPH && !pdev->tph_ext_support)
+		return -EINVAL;
 
-	pdev->tph_req_type = pdev->tph_ext_support ? PCI_TPH_REQ_EXT_TPH :
-						     PCI_TPH_REQ_TPH_ONLY;
+	pdev->tph_mode = mode;
+	pdev->tph_req_type = req_type;
 
 	/* Write them into TPH control register */
 	pci_read_config_dword(pdev, pdev->tph_cap + PCI_TPH_CTRL, &reg);
@@ -417,8 +402,51 @@ int pcie_enable_tph(struct pci_dev *pdev, int mode)
 
 	return 0;
 }
+
+/**
+ * pcie_enable_tph - Enable TPH support for device using a specific ST mode
+ * @pdev: PCI device
+ * @mode: ST mode to enable. Current supported modes include:
+ *
+ *   - PCI_TPH_ST_NS_MODE: NO ST Mode
+ *   - PCI_TPH_ST_IV_MODE: Interrupt Vector Mode
+ *   - PCI_TPH_ST_DS_MODE: Device Specific Mode
+ *
+ * Check whether the mode is actually supported by the device before enabling
+ * and return an error if not. Additionally determine what types of requests,
+ * TPH or extended TPH, can be issued by the device based on its TPH requester
+ * capability and the Root Port's completer capability.
+ *
+ * Return: 0 on success, otherwise negative value (-errno)
+ */
+int pcie_enable_tph(struct pci_dev *pdev, int mode)
+{
+	u8 req_type = pdev->tph_ext_support ? PCI_TPH_REQ_EXT_TPH :
+					      PCI_TPH_REQ_TPH_ONLY;
+	return enable_tph(pdev, mode, req_type);
+}
 EXPORT_SYMBOL(pcie_enable_tph);
 
+/**
+ * pcie_enable_tph_explicit - Enable TPH with explicit requester selection
+ * @pdev: PCI device to operate
+ * @mode: ST table operating mode (NS/IV/DS)
+ * @extended: true = EXT_TPH, false = standard TPH only
+ *
+ * Unlike auto-detecting pcie_enable_tph(), caller selects requester type
+ * manually instead of hardware auto-selection. Rejects EXT_TPH request
+ * if device lacks extended requester capability.
+ *
+ * Return: 0 on success, negative errno on failure.
+ */
+int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode, bool extended)
+{
+	u8 req_type = extended ? PCI_TPH_REQ_EXT_TPH : PCI_TPH_REQ_TPH_ONLY;
+
+	return enable_tph(pdev, mode, req_type);
+}
+EXPORT_SYMBOL(pcie_enable_tph_explicit);
+
 void pci_restore_tph_state(struct pci_dev *pdev)
 {
 	struct pci_cap_saved_state *save_state;
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 6f02b020d7d7..ca0faa98afac 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -29,6 +29,7 @@ int pcie_tph_get_cpu_st(struct pci_dev *dev,
 			unsigned int cpu, u16 *tag);
 void pcie_disable_tph(struct pci_dev *pdev);
 int pcie_enable_tph(struct pci_dev *pdev, int mode);
+int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode, bool extended);
 u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
 #else
@@ -42,6 +43,9 @@ static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
 static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
 { return -EINVAL; }
+static inline int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode,
+					   bool extended)
+{ return -EINVAL; }
 static inline u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
 { return 0; }
 static inline u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
-- 
2.17.1

[PATCH v19 05/18] PCI/TPH: Refactor pcie_tph_get_cpu_st & add explicit variant

[Chengwen Feng]

Refactor pcie_tph_get_cpu_st(): extract core logic into static internal
get_cpu_st() helper accepting explicit requester type parameter.

- Preserve original pcie_tph_get_cpu_st() unchanged as auto wrapper; it
  uses existing pdev->tph_req_type automatically, existing callers require
  no change.
- Add exported pcie_tph_get_cpu_st_explicit() with bool 'extended'
  parameter for manual STD/EXT requester selection, consumed by upcoming
  VFIO TPH code.
- Add capability check: reject explicit EXT request when device does not
  support extended TPH requester.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 68 ++++++++++++++++++++++++++++++-----------
 include/linux/pci-tph.h |  7 +++++
 2 files changed, 57 insertions(+), 18 deletions(-)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index c22eb7f4b0bd..70273c44e8fd 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -230,21 +230,8 @@ static int write_tag_to_st_table(struct pci_dev *pdev, int index, u16 tag)
 	return pci_write_config_word(pdev, offset, tag);
 }
 
-/**
- * pcie_tph_get_cpu_st() - Retrieve Steering Tag for a target memory associated
- * with a specific CPU
- * @pdev: PCI device
- * @mem_type: target memory type (volatile or persistent RAM)
- * @cpu: associated CPU id
- * @tag: Steering Tag to be returned
- *
- * Return the Steering Tag for a target memory that is associated with a
- * specific CPU as indicated by cpu.
- *
- * Return: 0 if success, otherwise negative value (-errno)
- */
-int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
-			unsigned int cpu, u16 *tag)
+static int get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
+		      u8 req_type, unsigned int cpu, u16 *tag)
 {
 #ifdef CONFIG_ACPI
 	struct pci_dev *rp;
@@ -268,19 +255,64 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
 		return -EINVAL;
 	}
 
-	*tag = tph_extract_tag(mem_type, pdev->tph_req_type, &info);
+	*tag = tph_extract_tag(mem_type, req_type, &info);
 
-	pci_dbg(pdev, "get steering tag: mem_type=%s, cpu=%d, tag=%#04x\n",
+	pci_dbg(pdev, "get steering tag: mem_type=%s, req_type=%u, cpu=%d, tag=%#04x\n",
 		(mem_type == TPH_MEM_TYPE_VM) ? "volatile" : "persistent",
-		cpu, *tag);
+		req_type, cpu, *tag);
 
 	return 0;
 #else
 	return -ENODEV;
 #endif
 }
+
+/**
+ * pcie_tph_get_cpu_st() - Retrieve Steering Tag for a target memory associated
+ * with a specific CPU
+ * @pdev: PCI device
+ * @mem_type: target memory type (volatile or persistent RAM)
+ * @cpu: associated CPU id
+ * @tag: Steering Tag to be returned
+ *
+ * Return the Steering Tag for a target memory that is associated with a
+ * specific CPU as indicated by cpu.
+ *
+ * Return: 0 if success, otherwise negative value (-errno)
+ */
+int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
+			unsigned int cpu, u16 *tag)
+{
+	return get_cpu_st(pdev, mem_type, pdev->tph_req_type, cpu, tag);
+}
 EXPORT_SYMBOL(pcie_tph_get_cpu_st);
 
+/**
+ * pcie_tph_get_cpu_st_explicit - Get ST with explicit requester type
+ * @pdev: PCI device
+ * @mem_type: target memory type (volatile or persistent RAM)
+ * @extended: true=EXT_TPH, false=standard TPH only
+ * @cpu: associated CPU id
+ * @tag: output steering tag pointer
+ *
+ * Unlike auto pcie_tph_get_cpu_st(), caller manually picks requester type.
+ * Rejects EXT request if device lacks extended requester capability.
+ *
+ * Return: 0 on success, negative errno on failure.
+ */
+int pcie_tph_get_cpu_st_explicit(struct pci_dev *pdev,
+				 enum tph_mem_type mem_type,
+				 bool extended, unsigned int cpu, u16 *tag)
+{
+	u8 req_type = extended ? PCI_TPH_REQ_EXT_TPH : PCI_TPH_REQ_TPH_ONLY;
+
+	if (extended && !pdev->tph_ext_support)
+		return -EINVAL;
+
+	return get_cpu_st(pdev, mem_type, req_type, cpu, tag);
+}
+EXPORT_SYMBOL(pcie_tph_get_cpu_st_explicit);
+
 /**
  * pcie_tph_set_st_entry() - Set Steering Tag in the ST table entry
  * @pdev: PCI device
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index ca0faa98afac..1a508b3d511f 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -27,6 +27,9 @@ int pcie_tph_set_st_entry(struct pci_dev *pdev,
 int pcie_tph_get_cpu_st(struct pci_dev *dev,
 			enum tph_mem_type mem_type,
 			unsigned int cpu, u16 *tag);
+int pcie_tph_get_cpu_st_explicit(struct pci_dev *pdev,
+				 enum tph_mem_type mem_type,
+				 bool extended, unsigned int cpu, u16 *tag);
 void pcie_disable_tph(struct pci_dev *pdev);
 int pcie_enable_tph(struct pci_dev *pdev, int mode);
 int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode, bool extended);
@@ -40,6 +43,10 @@ static inline int pcie_tph_get_cpu_st(struct pci_dev *dev,
 				      enum tph_mem_type mem_type,
 				      unsigned int cpu, u16 *tag)
 { return -EINVAL; }
+static inline int pcie_tph_get_cpu_st_explicit(struct pci_dev *pdev,
+				enum tph_mem_type mem_type,
+				bool extended, unsigned int cpu, u16 *tag)
+{ return -EINVAL; }
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
 static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
 { return -EINVAL; }
-- 
2.17.1

[PATCH v19 06/18] PCI/TPH: Expose the enabled TPH requester type

[Chengwen Feng]

From: Zhiping Zhang <zhipingz@meta.com>

Add pcie_tph_enabled_req_type() so drivers can query the enabled TPH
requester mode without reaching into pci_dev internals.

Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 12 ++++++++++++
 include/linux/pci-tph.h |  3 +++
 2 files changed, 15 insertions(+)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index 70273c44e8fd..ea777cc33ae6 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -479,6 +479,18 @@ int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode, bool extended)
 }
 EXPORT_SYMBOL(pcie_enable_tph_explicit);
 
+/**
+ * pcie_tph_enabled_req_type - Return the device's enabled TPH requester type
+ * @pdev: PCI device to query
+ *
+ * Return: PCI_TPH_REQ_DISABLE, PCI_TPH_REQ_TPH_ONLY or PCI_TPH_REQ_EXT_TPH.
+ */
+u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
+{
+	return pdev->tph_req_type;
+}
+EXPORT_SYMBOL(pcie_tph_enabled_req_type);
+
 void pci_restore_tph_state(struct pci_dev *pdev)
 {
 	struct pci_cap_saved_state *save_state;
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 1a508b3d511f..e4f7045fc152 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -33,6 +33,7 @@ int pcie_tph_get_cpu_st_explicit(struct pci_dev *pdev,
 void pcie_disable_tph(struct pci_dev *pdev);
 int pcie_enable_tph(struct pci_dev *pdev, int mode);
 int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode, bool extended);
+u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
 u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
 #else
@@ -47,6 +48,8 @@ static inline int pcie_tph_get_cpu_st_explicit(struct pci_dev *pdev,
 				enum tph_mem_type mem_type,
 				bool extended, unsigned int cpu, u16 *tag)
 { return -EINVAL; }
+static inline u8 pcie_tph_enabled_req_type(struct pci_dev *pdev)
+{ return PCI_TPH_REQ_DISABLE; }
 static inline void pcie_disable_tph(struct pci_dev *pdev) { }
 static inline int pcie_enable_tph(struct pci_dev *pdev, int mode)
 { return -EINVAL; }
-- 
2.17.1

[PATCH v19 07/18] PCI/TPH: Add pcie_tph_supported() helper to check TPH capability attributes

[Chengwen Feng]

Add new helper pcie_tph_supported() with want_ext parameter:
- want_ext = false: Check if device has valid TPH capability;
- want_ext = true: Check hardware Extended TPH support.

This helper is prepared for follow-up VFIO TPH virtualization patches to
uniformly query basic TPH existence and Extended TPH capability.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 19 +++++++++++++++++++
 include/linux/pci-tph.h |  3 +++
 2 files changed, 22 insertions(+)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index ea777cc33ae6..c080cf5401a3 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -596,3 +596,22 @@ void pci_tph_init(struct pci_dev *pdev)
 	save_size = sizeof(u32) + num_entries * sizeof(u16);
 	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);
 }
+
+/**
+ * pcie_tph_supported - Check TPH capability attribute
+ * @pdev: PCI device to query
+ * @want_ext: false - check TPH cap exists; true - check EXT_TPH support
+ *
+ * Return: true on matched condition, false otherwise
+ */
+bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext)
+{
+	if (!pdev->tph_cap)
+		return false;
+
+	if (!want_ext)
+		return true;
+
+	return pdev->tph_ext_support;
+}
+EXPORT_SYMBOL(pcie_tph_supported);
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index e4f7045fc152..5917a0694c1d 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -36,6 +36,7 @@ int pcie_enable_tph_explicit(struct pci_dev *pdev, int mode, bool extended);
 u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
 u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
+bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext);
 #else
 static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
 					unsigned int index, u16 tag)
@@ -60,6 +61,8 @@ static inline u16 pcie_tph_get_st_table_size(struct pci_dev *pdev)
 { return 0; }
 static inline u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
 { return PCI_TPH_LOC_NONE; }
+static inline bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext)
+{ return false; }
 #endif
 
 #endif /* LINUX_PCI_TPH_H */
-- 
2.17.1

[PATCH v19 08/18] PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM

[Chengwen Feng]

Add new helper pci_tph_dsm_supported() to check whether the root port
of a given PCI device exposes the TPH ST ACPI _DSM method.

Wrap all ACPI logic under CONFIG_ACPI guard, return false directly
when ACPI is disabled.

This helper is prepared for follow-up VFIO TPH virtualization patches
to uniformly judge platform CPU TPH resolving support.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/pci/tph.c       | 28 ++++++++++++++++++++++++++++
 include/linux/pci-tph.h |  3 +++
 2 files changed, 31 insertions(+)

diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
index c080cf5401a3..fcdba135a4f3 100644
--- a/drivers/pci/tph.c
+++ b/drivers/pci/tph.c
@@ -615,3 +615,31 @@ bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext)
 	return pdev->tph_ext_support;
 }
 EXPORT_SYMBOL(pcie_tph_supported);
+
+/**
+ * pcie_tph_dsm_supported - Check if root port exposes TPH ST _DSM
+ * @pdev: target PCI device
+ *
+ * Return true if root port of @pdev provides TPH ST DSM function,
+ * false otherwise (ACPI disabled or DSM missing).
+ */
+bool pcie_tph_dsm_supported(struct pci_dev *pdev)
+{
+#ifdef CONFIG_ACPI
+	struct pci_dev *rp = pcie_find_root_port(pdev);
+	acpi_handle handle;
+
+	if (!rp || !rp->bus || !rp->bus->bridge)
+		return false;
+
+	handle = ACPI_HANDLE(rp->bus->bridge);
+	if (!handle)
+		return false;
+
+	return acpi_check_dsm(handle, &pci_acpi_dsm_guid, 7,
+			      BIT(TPH_ST_DSM_FUNC_INDEX));
+#else
+	return false;
+#endif
+}
+EXPORT_SYMBOL_GPL(pcie_tph_dsm_supported);
diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h
index 5917a0694c1d..ac9692d740a6 100644
--- a/include/linux/pci-tph.h
+++ b/include/linux/pci-tph.h
@@ -37,6 +37,7 @@ u8 pcie_tph_enabled_req_type(struct pci_dev *pdev);
 u16 pcie_tph_get_st_table_size(struct pci_dev *pdev);
 u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev);
 bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext);
+bool pcie_tph_dsm_supported(struct pci_dev *pdev);
 #else
 static inline int pcie_tph_set_st_entry(struct pci_dev *pdev,
 					unsigned int index, u16 tag)
@@ -63,6 +64,8 @@ static inline u32 pcie_tph_get_st_table_loc(struct pci_dev *pdev)
 { return PCI_TPH_LOC_NONE; }
 static inline bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext)
 { return false; }
+static inline bool pcie_tph_dsm_supported(struct pci_dev *pdev)
+{ return false; }
 #endif
 
 #endif /* LINUX_PCI_TPH_H */
-- 
2.17.1

[PATCH v19 09/18] vfio/pci: Hide TPH capability when TPH is unsupported

[Chengwen Feng]

Check the device negotiated TPH support status before parsing the TPH
extended capability. Return zero length to hide the capability from
userspace if TPH is disabled during topology negotiation.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_config.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index a10ed733f0e3..5c6ab172df6c 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -22,6 +22,7 @@
 
 #include <linux/fs.h>
 #include <linux/pci.h>
+#include <linux/pci-tph.h>
 #include <linux/uaccess.h>
 #include <linux/vfio.h>
 #include <linux/slab.h>
@@ -1450,6 +1451,8 @@ static int vfio_ext_cap_len(struct vfio_pci_core_device *vdev, u16 ecap, u16 epo
 		byte &= PCI_DPA_CAP_SUBSTATE_MASK;
 		return PCI_DPA_BASE_SIZEOF + byte + 1;
 	case PCI_EXT_CAP_ID_TPH:
+		if (!pcie_tph_supported(pdev, false))
+			return 0;
 		ret = pci_read_config_dword(pdev, epos + PCI_TPH_CAP, &dword);
 		if (ret)
 			return pcibios_err_to_errno(ret);
-- 
2.17.1

[PATCH v19 10/18] vfio/pci: Introduce tph policy parameter for staged TPH feature enablement

[Chengwen Feng]

Add a module parameter 'tph' to implement staged TPH feature enabling
policy. The parameter accepts a value from 0 to 3 to progressively enable
TPH capabilities:

0 = No-ST mode: RESOLVE only returns PH for DMABUF source;
                ST programming unavailable
1 = Add IV mode: RESOLVE only returns PH for DMABUF source;
                 ST program supports NONE/DMABUF/CPU sources
2 = Add DS mode: RESOLVE returns PH+ST for DMABUF/CPU sources;
                 ST program supports NONE/DMABUF/CPU sources
3 = Add LITERAL: RESOLVE returns PH+ST for DMABUF/CPU sources;
                 ST program supports NONE/DMABUF/CPU/LITERAL sources

This commit only adds parameter plumbing and state storage, no
functional TPH logic is introduced yet.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci.c      | 25 +++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h |  7 +++++++
 include/linux/vfio_pci_core.h    |  1 +
 3 files changed, 33 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 830369ff878d..7ef82b445722 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -60,6 +60,28 @@ static bool disable_denylist;
 module_param(disable_denylist, bool, 0444);
 MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users.");
 
+static unsigned int tph_policy;
+static int tph_set(const char *val, const struct kernel_param *kp)
+{
+	return param_set_uint_minmax(val, kp, VFIO_PCI_TPH_POLICY_NO_ST,
+				     VFIO_PCI_TPH_POLICY_LITERAL);
+}
+static const struct kernel_param_ops tph_param_ops = {
+	.set = tph_set,
+	.get = param_get_uint,
+};
+module_param_cb(tph, &tph_param_ops, &tph_policy, 0644);
+MODULE_PARM_DESC(tph,
+	"Global TPH policy level (0=default No-ST):\n"
+	"0 = No-ST mode: RESOLVE only returns PH for DMABUF source;\n"
+	"                ST programming unavailable\n"
+	"1 = Add IV mode: RESOLVE only returns PH for DMABUF source;\n"
+	"                 ST program supports NONE/DMABUF/CPU sources\n"
+	"2 = Add DS mode: RESOLVE returns PH+ST for DMABUF/CPU sources;\n"
+	"                 ST program supports NONE/DMABUF/CPU sources\n"
+	"3 = Add LITERAL: RESOLVE returns PH+ST for DMABUF/CPU sources;\n"
+	"                 ST program supports NONE/DMABUF/CPU/LITERAL sources\n");
+
 static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev)
 {
 	switch (pdev->vendor) {
@@ -142,6 +164,9 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev)
 #ifdef CONFIG_VFIO_PCI_VGA
 	vdev->disable_vga = disable_vga;
 #endif
+#ifdef CONFIG_PCIE_TPH
+	vdev->tph_policy = tph_policy;
+#endif
 
 	return vfio_pci_core_init_dev(core_vdev);
 }
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index fca9d0dfac90..c997cc9bf330 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -11,6 +11,13 @@
 /* Cap maximum number of ioeventfds per device (arbitrary) */
 #define VFIO_PCI_IOEVENTFD_MAX		1000
 
+enum vfio_pci_tph_policy {
+	VFIO_PCI_TPH_POLICY_NO_ST = 0,
+	VFIO_PCI_TPH_POLICY_IV_ST = 1,
+	VFIO_PCI_TPH_POLICY_DS_ST = 2,
+	VFIO_PCI_TPH_POLICY_LITERAL = 3,
+};
+
 struct vfio_pci_ioeventfd {
 	struct list_head	next;
 	struct vfio_pci_core_device	*vdev;
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9a1674c152aa..a0641286dd90 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -149,6 +149,7 @@ struct vfio_pci_core_device {
 	struct notifier_block	nb;
 	struct rw_semaphore	memory_lock;
 	struct list_head	dmabufs;
+	u8			tph_policy;
 };
 
 enum vfio_pci_io_width {
-- 
2.17.1

[PATCH v19 11/18] vfio/pci: Virtualize PCIe TPH capability registers

[Chengwen Feng]

Implement virtualization and policy masking for PCIe TPH extended
capability:
- Split TPH config space permissions: keep header read-only, mark
  TPH_CTRL and ST-table entries virtually writable.
- Adjust virtual TPH capability bits according to hardware capability
  and tph_policy, hiding unsupported modes.
- Silently discard all write operations.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_config.c | 72 ++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 5c6ab172df6c..06d7b2fbf866 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1086,6 +1086,73 @@ static int __init init_pci_ext_cap_pwr_perm(struct perm_bits *perm)
 	return 0;
 }
 
+/* Permissions for TPH extended capability */
+static int __init init_pci_ext_cap_tph_perm(struct perm_bits *perm)
+{
+	int i;
+
+	if (alloc_perm_bits(perm, pci_ext_cap_length[PCI_EXT_CAP_ID_TPH]))
+		return -ENOMEM;
+
+	p_setd(perm, 0, ALL_VIRT, NO_WRITE);
+	p_setd(perm, PCI_TPH_CAP, ALL_VIRT, NO_WRITE);
+
+	p_setd(perm, PCI_TPH_CTRL, ALL_VIRT,
+	       PCI_TPH_CTRL_MODE_SEL_MASK | PCI_TPH_CTRL_REQ_EN_MASK);
+
+	/* Per PCI specification: There is an upper limit of 64 entries
+	 * when the ST table is located in the TPH Requester Extended
+	 * Capability structure.
+	 * And the pci_ext_cap_length[PCI_EXT_CAP_ID_TPH] is 0xFF, so the
+	 * following operation is fine.
+	 */
+	for (i = 0; i < 64; i++)
+		p_setw(perm, PCI_TPH_BASE_SIZEOF + i * sizeof(u16),
+		       (u16)ALL_VIRT, (u16)ALL_WRITE);
+
+	return 0;
+}
+
+static void vfio_tph_capability_adjust(struct vfio_pci_core_device *vdev,
+				       int pos)
+{
+	__le32 *vptr = (__le32 *)&vdev->vconfig[pos + PCI_TPH_CAP];
+	struct pci_dev *pdev = vdev->pdev;
+	u32 val = le32_to_cpu(*vptr);
+	bool need_adjust = false;
+
+	if (!pcie_tph_supported(pdev, true)) {
+		/* Remove extend TPH if root-port don't support */
+		val &= ~PCI_TPH_CAP_EXT_TPH;
+		need_adjust = true;
+	}
+
+	if (vdev->tph_policy == VFIO_PCI_TPH_POLICY_NO_ST) {
+		/* Report only No-ST mode supported */
+		val &= ~(PCI_TPH_CAP_ST_IV | PCI_TPH_CAP_ST_DS |
+			 PCI_TPH_CAP_LOC_MASK | PCI_TPH_CAP_ST_MASK);
+		need_adjust = true;
+	} else if (vdev->tph_policy == VFIO_PCI_TPH_POLICY_IV_ST) {
+		/* Report only No-ST and IV modes supported */
+		val &= ~PCI_TPH_CAP_ST_DS;
+		/* Remove ST location and size if dev don't support IV mode */
+		if (!(val & PCI_TPH_CAP_ST_IV))
+			val &= ~(PCI_TPH_CAP_LOC_MASK | PCI_TPH_CAP_ST_MASK);
+		need_adjust = true;
+	}
+
+	if (need_adjust)
+		*vptr = cpu_to_le32(val);
+}
+
+static int vfio_find_cap_start(struct vfio_pci_core_device *vdev, int pos);
+static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
+				 int count, struct perm_bits *perm,
+				 int offset, __le32 val)
+{
+	return count;
+}
+
 /*
  * Initialize the shared permission tables
  */
@@ -1101,6 +1168,7 @@ void vfio_pci_uninit_perm_bits(void)
 
 	free_perm_bits(&ecap_perms[PCI_EXT_CAP_ID_ERR]);
 	free_perm_bits(&ecap_perms[PCI_EXT_CAP_ID_PWR]);
+	free_perm_bits(&ecap_perms[PCI_EXT_CAP_ID_TPH]);
 }
 
 int __init vfio_pci_init_perm_bits(void)
@@ -1121,6 +1189,8 @@ int __init vfio_pci_init_perm_bits(void)
 	/* Extended capabilities */
 	ret |= init_pci_ext_cap_err_perm(&ecap_perms[PCI_EXT_CAP_ID_ERR]);
 	ret |= init_pci_ext_cap_pwr_perm(&ecap_perms[PCI_EXT_CAP_ID_PWR]);
+	ret |= init_pci_ext_cap_tph_perm(&ecap_perms[PCI_EXT_CAP_ID_TPH]);
+	ecap_perms[PCI_EXT_CAP_ID_TPH].writefn = vfio_tph_config_write;
 	ecap_perms[PCI_EXT_CAP_ID_VNDR].writefn = vfio_raw_config_write;
 	ecap_perms[PCI_EXT_CAP_ID_DVSEC].writefn = vfio_raw_config_write;
 
@@ -1704,6 +1774,8 @@ static int vfio_ecap_init(struct vfio_pci_core_device *vdev)
 		ret = vfio_fill_vconfig_bytes(vdev, epos, len);
 		if (ret)
 			return ret;
+		if (ecap == PCI_EXT_CAP_ID_TPH && !hidden)
+			vfio_tph_capability_adjust(vdev, epos);
 
 		/*
 		 * If we're just using this capability to anchor the list,
-- 
2.17.1

[PATCH v19 12/18] vfio/pci: Add dmabuf TPH metadata storage and fd query helper

[Chengwen Feng]

Add TPH metadata fields to struct vfio_pci_dma_buf to store PH,
basic/extended ST and their validity flags.

Implement vfio_pci_dma_buf_get_pci_tph() to fetch TPH tag and PH from
dmabuf, distinguish basic and extended ST namespace.

Add vfio_pci_dma_buf_get_tph_by_fd() to resolve TPH data via dmabuf fd,
validate dmabuf owner ops. Provide stub implementation if
CONFIG_VFIO_PCI_DMABUF is unset.

Lay dmabuf foundation for the upcoming TPH_RESOLVE feature.

Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_dmabuf.c | 49 +++++++++++++++++++++++++++++-
 drivers/vfio/pci/vfio_pci_priv.h   |  6 ++++
 2 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
index c16f460c01d6..d9c5e434e586 100644
--- a/drivers/vfio/pci/vfio_pci_dmabuf.c
+++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
@@ -19,7 +19,14 @@ struct vfio_pci_dma_buf {
 	u32 nr_ranges;
 	struct kref kref;
 	struct completion comp;
-	u8 revoked : 1;
+
+	/* Protected by dmabuf->resv. */
+	u8 revoked:1;
+	u8 tph_ph:2;
+	u8 tph_st_valid:1;
+	u8 tph_xst_valid:1;
+	u8 tph_st;
+	u16 tph_xst;
 };
 
 static int vfio_pci_dma_buf_attach(struct dma_buf *dmabuf,
@@ -81,6 +88,26 @@ static void vfio_pci_dma_buf_unmap(struct dma_buf_attachment *attachment,
 	kref_put(&priv->kref, vfio_pci_dma_buf_done);
 }
 
+static int vfio_pci_dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
+					u16 *tag, u8 *ph)
+{
+	struct vfio_pci_dma_buf *priv = dmabuf->priv;
+
+	dma_resv_assert_held(priv->dmabuf->resv);
+
+	if (extended) {
+		if (!priv->tph_xst_valid)
+			return -EOPNOTSUPP;
+		*tag = priv->tph_xst;
+	} else {
+		if (!priv->tph_st_valid)
+			return -EOPNOTSUPP;
+		*tag = priv->tph_st;
+	}
+	*ph = priv->tph_ph;
+	return 0;
+}
+
 static void vfio_pci_dma_buf_release(struct dma_buf *dmabuf)
 {
 	struct vfio_pci_dma_buf *priv = dmabuf->priv;
@@ -106,6 +133,26 @@ static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
 	.release = vfio_pci_dma_buf_release,
 };
 
+int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended, u16 *tag, u8 *ph)
+{
+	struct dma_buf *dmabuf;
+	int ret = 0;
+
+	dmabuf = dma_buf_get(fd);
+	if (IS_ERR(dmabuf))
+		return PTR_ERR(dmabuf);
+
+	if (dmabuf->ops != &vfio_pci_dmabuf_ops) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);
+out:
+	dma_buf_put(dmabuf);
+	return ret;
+}
+
 /*
  * This is a temporary "private interconnect" between VFIO DMABUF and iommufd.
  * It allows the two co-operating drivers to exchange the physical address of
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index c997cc9bf330..d7bc52f488c6 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -122,12 +122,18 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pdev)
 }
 
 #ifdef CONFIG_VFIO_PCI_DMABUF
+int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended, u16 *tag, u8 *ph);
 int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 				  struct vfio_device_feature_dma_buf __user *arg,
 				  size_t argsz);
 void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev);
 void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked);
 #else
+static inline int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended,
+						 u16 *tag, u8 *ph)
+{
+	return -EOPNOTSUPP;
+}
 static inline int
 vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 			      struct vfio_device_feature_dma_buf __user *arg,
-- 
2.17.1

[PATCH v19 13/18] vfio/pci: Introduce VFIO_DEVICE_FEATURE_TPH family uapi for PCI TPH control

[Chengwen Feng]

Add user API definitions for PCI TPH device feature set, including three
feature IDs:
- VFIO_DEVICE_FEATURE_TPH: Global TPH enable and capability query
- VFIO_DEVICE_FEATURE_TPH_RESOLVE: Resolve PH/ST from specified source
  (GET only)
- VFIO_DEVICE_FEATURE_TPH_ST: Batch program ST table entries (SET only)

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 include/uapi/linux/vfio.h | 62 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 5de618a3a5ee..ed04b6280cd6 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -1534,6 +1534,68 @@ struct vfio_device_feature_dma_buf {
  */
 #define VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2  12
 
+/* PCIe TPH device feature definitions for VFIO_DEVICE_FEATURE ioctl */
+#define VFIO_DEVICE_FEATURE_TPH		13
+#define VFIO_DEVICE_FEATURE_TPH_RESOLVE	14
+#define VFIO_DEVICE_FEATURE_TPH_ST	15
+
+/*
+ * VFIO_DEVICE_FEATURE_TPH - Control and query PCI TPH capabilities
+ *
+ * SET: Opt-in to TPH feature; flags must be zero.
+ * GET: Return supported TPH capability bits in flags.
+ */
+struct vfio_device_feature_tph {
+	__u32 flags;
+};
+
+/* Capability bits returned by GET on VFIO_DEVICE_FEATURE_TPH */
+#define VFIO_DEVICE_TPH_CAP_DMABUF	(1u << 0) /* DMABUF sources resolvable */
+#define VFIO_DEVICE_TPH_CAP_CPU		(1u << 1) /* CPU sources resolvable via _DSM */
+#define VFIO_DEVICE_TPH_CAP_LITERAL	(1u << 2) /* LITERAL ST source supported */
+
+/*
+ * VFIO_DEVICE_FEATURE_TPH_RESOLVE - Resolve PH/ST tag value from source
+ * Note: GET only
+ */
+struct vfio_device_feature_tph_resolve {
+	__u32 flags;	/* IN: source type + namespace modifiers */
+	__u32 src;	/* IN: dma-buf fd or CPU ID based on flags */
+	__u8 valid;	/* OUT: bitmap indicating valid output fields */
+#define VFIO_DEVICE_TPH_VALID_PH (1u << 0) /* ph holds valid processing hint */
+#define VFIO_DEVICE_TPH_VALID_ST (1u << 1) /* st holds valid steering tag */
+	__u8 ph;	/* OUT: TPH processing hint */
+	__u16 st;	/* OUT: raw steering tag value */
+};
+
+/*
+ * VFIO_DEVICE_FEATURE_TPH_ST - Batch program architected ST table entries
+ * Return value: number of successfully written entries
+ * Note: SET only
+ */
+struct vfio_device_feature_tph_st {
+	__u32 flags;		/* IN: source type + namespace modifiers */
+	__u16 start;		/* IN: first ST table index / MSI-X vector */
+	__u16 count;		/* IN: number of contiguous entries to program */
+	__aligned_u64 dests;	/* IN: user pointer to array of source u32 values */
+};
+
+/* Common flags shared by TPH_RESOLVE and TPH_ST features */
+#define VFIO_DEVICE_TPH_SRC_NONE           (1u << 0) /* TPH_ST only: set ST=0 */
+#define VFIO_DEVICE_TPH_SRC_DMABUF	   (1u << 1) /* src = dma-buf fd */
+#define VFIO_DEVICE_TPH_SRC_CPU_VOLATILE   (1u << 2) /* src = volatile memory CPU ID */
+#define VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT (1u << 3) /* src = persistent memory CPU ID */
+#define VFIO_DEVICE_TPH_SRC_LITERAL        (1u << 4) /* TPH_ST only: src = raw literal ST */
+#define VFIO_DEVICE_TPH_EXTENDED           (1u << 30) /* Use extended 16-bit ST namespace */
+#define VFIO_DEVICE_TPH_REQUIRE_ST         (1u << 31) /* TPH_ST only: fail if resolved ST=0 */
+
+/* Mask for mutually exclusive source type bits, only one may be set */
+#define VFIO_DEVICE_TPH_SRC_MASK	(VFIO_DEVICE_TPH_SRC_NONE | \
+					 VFIO_DEVICE_TPH_SRC_DMABUF | \
+					 VFIO_DEVICE_TPH_SRC_CPU_VOLATILE | \
+					 VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT | \
+					 VFIO_DEVICE_TPH_SRC_LITERAL)
+
 /* -------- API for Type1 VFIO IOMMU -------- */
 
 /**
-- 
2.17.1

[PATCH v19 14/18] vfio/pci: Implement VFIO_DEVICE_FEATURE_TPH and valid TPH config write support

[Chengwen Feng]

Implement vfio_pci_core_feature_tph() to handle VFIO_DEVICE_FEATURE_TPH:
- SET permits TPH feature usage for the device;
- GET reports available TPH capabilities according to tph_policy and
  root-port DSM support.

Previously VFIO dropped all writes to the TPH capability register.
Now implement vfio_tph_config_write() to support legitimate TPH_CTRL
modification after TPH feature is permitted.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_config.c | 54 ++++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c   | 36 ++++++++++++++++++++
 include/linux/vfio_pci_core.h      |  1 +
 3 files changed, 91 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 06d7b2fbf866..388dd6fed16b 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1150,6 +1150,60 @@ static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
 				 int count, struct perm_bits *perm,
 				 int offset, __le32 val)
 {
+	u16 start = vfio_find_cap_start(vdev, pos);
+	struct pci_dev *pdev = vdev->pdev;
+	u32 org_ctrl, new_ctrl, cap;
+	u8 mode, req, org_req;
+	__le32 org_val = 0;
+	bool extended;
+	int ret;
+
+	if (!vdev->tph_permit)
+		return count;
+
+	down_write(&vdev->memory_lock);
+
+	org_ctrl = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CTRL]);
+	vfio_default_config_read(vdev, pos, count, perm, offset, &org_val);
+
+	ret = vfio_default_config_write(vdev, pos, count, perm, offset, val);
+	if (ret != count)
+		goto out;
+
+	new_ctrl = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CTRL]);
+	if (new_ctrl == org_ctrl)
+		goto out; /* Only care about changes in TPH_CTRL. */
+
+	cap = le32_to_cpu(*(__le32 *)&vdev->vconfig[start + PCI_TPH_CAP]);
+	mode = FIELD_GET(PCI_TPH_CTRL_MODE_SEL_MASK, new_ctrl);
+	req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, new_ctrl);
+	if (mode > PCI_TPH_ST_DS_MODE || !(cap & (1u << mode)) || req == 0x2 ||
+		(req == PCI_TPH_REQ_EXT_TPH && !(cap & PCI_TPH_CAP_EXT_TPH)))
+		goto restore; /* Drop invalid or unsupported write value */
+
+	org_req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, org_ctrl);
+	if (req == org_req)
+		goto out; /* Only care about requester enable */
+
+	ret = vfio_pci_set_power_state(vdev, PCI_D0);
+	if (ret)
+		goto restore; /* Drop this write */
+
+	if (req == PCI_TPH_REQ_TPH_ONLY || req == PCI_TPH_REQ_EXT_TPH) {
+		extended = !!(req == PCI_TPH_REQ_EXT_TPH);
+		ret = pcie_enable_tph_explicit(pdev, mode, extended);
+		if (ret)
+			goto restore;
+	} else if (req == PCI_TPH_REQ_DISABLE) {
+		pcie_disable_tph(vdev->pdev);
+	}
+
+	goto out;
+
+restore:
+	vfio_default_config_write(vdev, pos, count, perm, offset, org_val);
+out:
+	up_write(&vdev->memory_lock);
 	return count;
 }
 
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 3f11a9624b9c..14944d3ea86e 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -30,6 +30,7 @@
 #include <linux/sched/mm.h>
 #include <linux/iommufd.h>
 #include <linux/pci-p2pdma.h>
+#include <linux/pci-tph.h>
 #include <linux/seq_file.h>
 #if IS_ENABLED(CONFIG_EEH)
 #include <asm/eeh.h>
@@ -610,6 +611,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 		goto out_disable_device;
 
 	vdev->reset_works = !ret;
+	vdev->tph_permit = false;
 	pci_save_state(pdev);
 	vdev->pci_saved_state = pci_store_saved_state(pdev);
 	if (!vdev->pci_saved_state)
@@ -1607,6 +1609,38 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
 	return 0;
 }
 
+static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
+				     u32 flags,
+				     struct vfio_device_feature_tph __user *arg,
+				     size_t argsz)
+{
+	struct vfio_device_feature_tph tph = {0};
+	int ret;
+
+	if (!pcie_tph_supported(vdev->pdev, false))
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz,
+			VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_SET,
+			sizeof(tph));
+	if (ret <= 0)
+		return ret;
+
+	if (flags & VFIO_DEVICE_FEATURE_SET) {
+		vdev->tph_permit = 1;
+		return 0;
+	}
+
+	tph.flags = VFIO_DEVICE_TPH_CAP_DMABUF;
+	if (vdev->tph_policy != VFIO_PCI_TPH_POLICY_NO_ST &&
+		pcie_tph_dsm_supported(vdev->pdev))
+		tph.flags |= VFIO_DEVICE_TPH_CAP_CPU;
+	if (vdev->tph_policy == VFIO_PCI_TPH_POLICY_LITERAL)
+		tph.flags |= VFIO_DEVICE_TPH_CAP_LITERAL;
+
+	return copy_to_user(arg, &tph, sizeof(tph)) ? -EFAULT : 0;
+}
+
 int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 				void __user *arg, size_t argsz)
 {
@@ -1625,6 +1659,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 		return vfio_pci_core_feature_token(vdev, flags, arg, argsz);
 	case VFIO_DEVICE_FEATURE_DMA_BUF:
 		return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz);
+	case VFIO_DEVICE_FEATURE_TPH:
+		return vfio_pci_core_feature_tph(vdev, flags, arg, argsz);
 	default:
 		return -ENOTTY;
 	}
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index a0641286dd90..9a6e82c5c00c 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -150,6 +150,7 @@ struct vfio_pci_core_device {
 	struct rw_semaphore	memory_lock;
 	struct list_head	dmabufs;
 	u8			tph_policy;
+	bool			tph_permit;
 };
 
 enum vfio_pci_io_width {
-- 
2.17.1

[PATCH v19 15/18] vfio/pci: Implement TPH_RESOLVE feature for DMABUF and CPU source resolving

[Chengwen Feng]

Add vfio_pci_core_feature_tph_resolve() to implement the
VFIO_DEVICE_FEATURE_TPH_RESOLVE GET-only uAPI.

Resolve PH/ST values from two supported sources:
- DMABUF fd: retrieve stored TPH metadata via dmabuf helper
- CPU ID: fetch ST tag from platform via PCIe TPH core helper

Gate supported source types dynamically according to the tph_policy level:
No-ST/IV modes only permit DMABUF resolution, while DS+ modes additionally
allow CPU volatile/persistent sources.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 75 ++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 14944d3ea86e..ed6bbffa8b26 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1641,6 +1641,78 @@ static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
 	return copy_to_user(arg, &tph, sizeof(tph)) ? -EFAULT : 0;
 }
 
+static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
+			u32 flags,
+			struct vfio_device_feature_tph_resolve __user *arg,
+			size_t argsz)
+{
+	const u32 permit_src_mask[] = {
+		[VFIO_PCI_TPH_POLICY_NO_ST] = VFIO_DEVICE_TPH_SRC_DMABUF,
+		[VFIO_PCI_TPH_POLICY_IV_ST] = VFIO_DEVICE_TPH_SRC_DMABUF,
+		[VFIO_PCI_TPH_POLICY_DS_ST ... VFIO_PCI_TPH_POLICY_LITERAL] =
+					VFIO_DEVICE_TPH_SRC_DMABUF |
+					VFIO_DEVICE_TPH_SRC_CPU_VOLATILE |
+					VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT,
+	};
+	u32 permit_flags = permit_src_mask[vdev->tph_policy] |
+			   VFIO_DEVICE_TPH_EXTENDED;
+	struct vfio_device_feature_tph_resolve resolve = {0};
+	enum tph_mem_type mtype;
+	bool extended;
+	u32 src_bits;
+	u16 tag = 0;
+	u8 ph = 0;
+	int ret;
+
+	if (!vdev->tph_permit)
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET,
+				 sizeof(resolve));
+	if (ret <= 0)
+		return ret;
+
+	if (copy_from_user(&resolve, arg, sizeof(resolve)))
+		return -EFAULT;
+
+	src_bits = resolve.flags & VFIO_DEVICE_TPH_SRC_MASK;
+	if (!(resolve.flags & permit_flags) || !is_power_of_2(src_bits))
+		return -EINVAL;
+	extended = !!(resolve.flags & VFIO_DEVICE_TPH_EXTENDED);
+	if (extended && !pcie_tph_supported(vdev->pdev, true))
+		return -EINVAL;
+
+	resolve.valid = 0;
+	resolve.ph = 0;
+	resolve.st = 0;
+
+	if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF) {
+		ret = vfio_pci_dma_buf_get_tph_by_fd(resolve.src, extended,
+						     &tag, &ph);
+		if (ret)
+			return ret;
+		resolve.ph = ph;
+		resolve.valid = VFIO_DEVICE_TPH_VALID_PH;
+		if (vdev->tph_policy >= VFIO_PCI_TPH_POLICY_DS_ST) {
+			resolve.st = tag;
+			resolve.valid |= VFIO_DEVICE_TPH_VALID_ST;
+		}
+		goto out;
+	}
+
+	mtype = (src_bits & VFIO_DEVICE_TPH_SRC_CPU_VOLATILE) ?
+			TPH_MEM_TYPE_VM : TPH_MEM_TYPE_PM;
+	ret = pcie_tph_get_cpu_st_explicit(vdev->pdev, mtype, extended,
+					   resolve.src, &tag);
+	if (ret)
+		return ret;
+	resolve.valid = VFIO_DEVICE_TPH_VALID_ST;
+	resolve.st = tag;
+
+out:
+	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
+}
+
 int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 				void __user *arg, size_t argsz)
 {
@@ -1661,6 +1733,9 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 		return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz);
 	case VFIO_DEVICE_FEATURE_TPH:
 		return vfio_pci_core_feature_tph(vdev, flags, arg, argsz);
+	case VFIO_DEVICE_FEATURE_TPH_RESOLVE:
+		return vfio_pci_core_feature_tph_resolve(vdev, flags,
+							 arg, argsz);
 	default:
 		return -ENOTTY;
 	}
-- 
2.17.1

[PATCH v19 16/18] vfio/pci: Implement TPH_ST feature for batch ST table programming

[Chengwen Feng]

Add vfio_pci_core_feature_tph_st() to implement the SET-only
VFIO_DEVICE_FEATURE_TPH_ST uAPI for batch TPH ST table programming.

Implement helper routines to calculate ST table size and resolve ST tags
from multiple sources: NONE, DMABUF, CPU volatile/persistent, and LITERAL.

Batch program contiguous ST table entries according to user inputs, support
stop-on-zero-ST semantics, and return the number of successfully written
entries. Hold memory_lock and ensure device is in D0 state during hardware
operations.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 119 +++++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index ed6bbffa8b26..7251e251824a 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1713,6 +1713,123 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
 	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
 }
 
+static u32 tph_calc_st_size(struct vfio_pci_core_device *vdev)
+{
+	struct pci_dev *pdev = vdev->pdev;
+	u32 loc = pcie_tph_get_st_table_loc(pdev);
+	int ret;
+
+	if (loc == PCI_TPH_LOC_CAP) {
+		return pcie_tph_get_st_table_size(pdev);
+	} else if (loc == PCI_TPH_LOC_MSIX) {
+		ret = pci_msix_vec_count(pdev);
+		if (ret < 0)
+			return 0;
+		return ret;
+	} else {
+		return 0;
+	}
+}
+
+static int tph_get_st_tag(struct pci_dev *pdev, u32 src_bits, u32 src_hndl,
+			  bool extended, u16 *tag)
+{
+	int ret = 0;
+	u8 ph;
+
+	if (src_bits & VFIO_DEVICE_TPH_SRC_NONE)
+		*tag = 0;
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF)
+		ret = vfio_pci_dma_buf_get_tph_by_fd(src_hndl, extended,
+						     tag, &ph);
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_VOLATILE)
+		ret = pcie_tph_get_cpu_st_explicit(pdev, TPH_MEM_TYPE_VM,
+					extended, src_hndl, tag);
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_PERSISTENT)
+		ret = pcie_tph_get_cpu_st_explicit(pdev, TPH_MEM_TYPE_PM,
+					extended, src_hndl, tag);
+	else if (src_bits & VFIO_DEVICE_TPH_SRC_LITERAL)
+		*tag = src_hndl;
+
+	if (ret != 0)
+		*tag = 0;
+
+	return ret;
+}
+
+static int vfio_pci_core_feature_tph_st(struct vfio_pci_core_device *vdev,
+				u32 flags,
+				struct vfio_device_feature_tph_st __user *arg,
+				size_t argsz)
+{
+	u32 permit_flags = VFIO_DEVICE_TPH_SRC_MASK | VFIO_DEVICE_TPH_EXTENDED;
+	struct vfio_device_feature_tph_st tph_st = {0};
+	struct pci_dev *pdev = vdev->pdev;
+	u32 src_bits, st_size;
+	bool stop_on_zero_st;
+	u32 *src_hndl = NULL;
+	void __user *uptr;
+	bool extended;
+	int ret, i;
+	u16 tag;
+
+	if (!vdev->tph_permit || vdev->tph_policy == VFIO_PCI_TPH_POLICY_NO_ST)
+		return -EOPNOTSUPP;
+
+	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET,
+				 sizeof(tph_st));
+	if (ret <= 0)
+		return ret;
+
+	if (copy_from_user(&tph_st, arg, sizeof(tph_st)))
+		return -EFAULT;
+
+	src_bits = tph_st.flags & VFIO_DEVICE_TPH_SRC_MASK;
+	if (vdev->tph_policy != VFIO_PCI_TPH_POLICY_LITERAL)
+		permit_flags &= ~VFIO_DEVICE_TPH_SRC_LITERAL;
+	if (!(tph_st.flags & permit_flags) || !is_power_of_2(src_bits))
+		return -EINVAL;
+	extended = !!(tph_st.flags & VFIO_DEVICE_TPH_EXTENDED);
+	if (extended && !pcie_tph_supported(vdev->pdev, true))
+		return -EINVAL;
+
+	st_size = tph_calc_st_size(vdev);
+	if (tph_st.start >= st_size || tph_st.count > st_size - tph_st.start ||
+		tph_st.count == 0)
+		return -EINVAL;
+
+	uptr = u64_to_user_ptr(tph_st.dests);
+	if (!(src_bits & VFIO_DEVICE_TPH_SRC_NONE)) {
+		src_hndl = memdup_array_user(uptr, tph_st.count, sizeof(u32));
+		if (IS_ERR(src_hndl))
+			return PTR_ERR(src_hndl);
+	}
+
+	down_write(&vdev->memory_lock);
+	ret = vfio_pci_set_power_state(vdev, PCI_D0);
+	if (ret)
+		goto out;
+
+	stop_on_zero_st = !!(tph_st.flags & VFIO_DEVICE_TPH_REQUIRE_ST);
+	if (tph_st.flags & VFIO_DEVICE_TPH_SRC_NONE)
+		stop_on_zero_st = false;
+	for (i = 0; i < tph_st.count; i++) {
+		ret = tph_get_st_tag(pdev, src_bits, src_hndl ? src_hndl[i] : 0,
+				     extended, &tag);
+		if (ret || (stop_on_zero_st && tag == 0))
+			break;
+		ret = pcie_tph_set_st_entry(pdev, tph_st.start + i, tag);
+		if (ret)
+			break;
+	}
+	ret = i;
+
+out:
+	up_write(&vdev->memory_lock);
+	kfree(src_hndl);
+	return ret;
+}
+
 int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 				void __user *arg, size_t argsz)
 {
@@ -1736,6 +1853,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
 	case VFIO_DEVICE_FEATURE_TPH_RESOLVE:
 		return vfio_pci_core_feature_tph_resolve(vdev, flags,
 							 arg, argsz);
+	case VFIO_DEVICE_FEATURE_TPH_ST:
+		return vfio_pci_core_feature_tph_st(vdev, flags, arg, argsz);
 	default:
 		return -ENOTTY;
 	}
-- 
2.17.1

[PATCH v19 17/18] vfio/pci: Reset hardware TPH state on device enable/disable

[Chengwen Feng]

VFIO allows user space to enable and modify PCI TPH state, but device
reset only clears hardware TPH registers, while the software TPH state
maintained in pci/tph.c remains stale.

Fix the stale state leakage by explicitly disabling TPH in core enable
and disable paths:
- In vfio_pci_core_enable(), clear leftover TPH state from previous
  user sessions before device initialization.
- In vfio_pci_core_disable(), revert any user-modified TPH state
  when releasing the device.

This ensures consistent TPH hardware and software state for each
user session and avoids residual state contamination.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 7251e251824a..5a14439d1ccb 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -612,6 +612,8 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 
 	vdev->reset_works = !ret;
 	vdev->tph_permit = false;
+	/* Reset TPH status on new user session */
+	pcie_disable_tph(vdev->pdev);
 	pci_save_state(pdev);
 	vdev->pci_saved_state = pci_store_saved_state(pdev);
 	if (!vdev->pci_saved_state)
@@ -759,6 +761,9 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 
 	vfio_pci_zdev_close_device(vdev);
 
+	/* Reset TPH status on exit user session */
+	pcie_disable_tph(vdev->pdev);
+
 	/*
 	 * If we have saved state, restore it.  If we can reset the device,
 	 * even better.  Resetting with current state seems better than
-- 
2.17.1

[PATCH v19 18/18] vfio/pci: Expose tph_policy via debugfs

[Chengwen Feng]

Add debugfs read-only file "tph_policy" to report per-device TPH policy
value.

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 5a14439d1ccb..9c0f3c7a1ef7 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -128,10 +128,19 @@ static int vfio_pci_core_debugfs_disable_idle_d3(struct seq_file *seq,
 	return 0;
 }
 
+static int vfio_pci_core_debugfs_tph_policy(struct seq_file *seq,
+					    void *data)
+{
+	struct vfio_pci_core_device *vdev = vfio_pci_core_debugfs_private(seq);
+
+	seq_printf(seq, "%u\n", vdev->tph_policy);
+	return 0;
+}
+
 /*
- * disable_idle_d3 and nointxmask are writable module parameters latched
- * per device at init, so a device's effective value can differ from the
- * current parameter setting.  Expose the per-device (read-only) values
+ * disable_idle_d3, nointxmask and tph_policy are writable module parameters
+ * latched per device at init, so a device's effective value can differ from
+ * the current parameter setting.  Expose the per-device (read-only) values
  * here for visibility; read-only parameters can't drift and are omitted.
  */
 static void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
@@ -147,6 +156,8 @@ static void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
 				    vfio_pci_core_debugfs_nointxmask);
 	debugfs_create_devm_seqfile(dev, "disable_idle_d3", pci_dir,
 				    vfio_pci_core_debugfs_disable_idle_d3);
+	debugfs_create_devm_seqfile(dev, "tph_policy", pci_dir,
+				    vfio_pci_core_debugfs_tph_policy);
 }
 #else
 static inline void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
-- 
2.17.1

Re: [PATCH v19 06/18] PCI/TPH: Expose the enabled TPH requester type

[sashiko-bot]

> From: Zhiping Zhang <zhipingz@meta.com>
> 
> Add pcie_tph_enabled_req_type() so drivers can query the enabled TPH
> requester mode without reaching into pci_dev internals.
> 
> Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=6

Re: [PATCH v19 10/18] vfio/pci: Introduce tph policy parameter for staged TPH feature enablement

[sashiko-bot]

> Add a module parameter 'tph' to implement staged TPH feature enabling
> policy. The parameter accepts a value from 0 to 3 to progressively enable
> TPH capabilities:
> 
> 0 = No-ST mode: RESOLVE only returns PH for DMABUF source;
>                 ST programming unavailable
> 1 = Add IV mode: RESOLVE only returns PH for DMABUF source;
>                  ST program supports NONE/DMABUF/CPU sources
> 2 = Add DS mode: RESOLVE returns PH+ST for DMABUF/CPU sources;
>                  ST program supports NONE/DMABUF/CPU sources
> 3 = Add LITERAL: RESOLVE returns PH+ST for DMABUF/CPU sources;
>                  ST program supports NONE/DMABUF/CPU/LITERAL sources
> 
> This commit only adds parameter plumbing and state storage, no
> functional TPH logic is introduced yet.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=10

Re: [PATCH v19 04/18] PCI/TPH: Refactor pcie_enable_tph & add explicit requester variant

[sashiko-bot]

> Refactor pcie_enable_tph implementation: extract core logic into static
> internal enable_tph() helper accepting explicit requester type.
> 
> - Preserve original pcie_enable_tph() unchanged as auto wrapper; it
>   auto-selects EXT/standard TPH requester per device capability, existing
>   bnxt/mlx5 callers require zero modification.
> - Add exported pcie_enable_tph_explicit() with bool 'extended' parameter
>   for explicit STD/EXT selection, used by upcoming VFIO TPH support.
> 
> Input validation for EXT_TPH availability is retained inside helper to
> reject invalid explicit EXT request if hardware does not support extended
> requester.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=4

Re: [PATCH v19 01/18] PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction

[sashiko-bot]

> pcie_tph_get_st_table_loc() incorrectly uses FIELD_GET(), which shifts the
> field value to bit 0. But the function is designed to return raw
> PCI_TPH_LOC_* values as defined in the function comment.
> 
> This causes incorrect ST table location detection. Fix it by using bitwise
> AND with PCI_TPH_CAP_LOC_MASK to return the unshifted field value matching
> the function specification.
> 
> This doesn't make a difference to mlx5_st_create(), the lone external
> caller, because it only checks for PCI_TPH_LOC_NONE (0), but will be needed
> for callers that check for PCI_TPH_LOC_CAP or PCI_TPH_LOC_MSIX.
> 
> Also add tph_cap validation for pcie_tph_get_st_table_loc() to prevent
> invalid PCI configuration space access when TPH is not supported. Add stub
> functions for pcie_tph_get_st_table_size() and pcie_tph_get_st_table_loc()
> when !CONFIG_PCIE_TPH.
> 
> Fixes: d2e8a34876ce ("PCI/TPH: Add Steering Tag support")
> Cc: stable@vger.kernel.org
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=1

Re: [PATCH v19 02/18] PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing

[sashiko-bot]

> Split tph_enabled from shared pci_dev bitfield into spare bit of tph_cap's
> u16: tph_cap is immutable post-enumeration (15 bits for offset), remaining
> 1 bit stores tph_enabled. Removes cross-bitfield concurrent write hazards
> highlighted by Sashiko after VFIO TPH exposure. No functional changes.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=2

Re: [PATCH v19 07/18] PCI/TPH: Add pcie_tph_supported() helper to check TPH capability attributes

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] pcie_tph_supported() does not check the system-wide pci_tph_disabled flag, leading to falsely reporting TPH support when it has been explicitly disabled by the administrator.
--

commit 089d399dcdf1ae5d6be4d557f772bf15c0d7d461
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Add pcie_tph_supported() helper to check TPH capability attributes

Add new helper pcie_tph_supported() to check if a device has a valid TPH
capability or hardware Extended TPH support. This helper is intended for
follow-up VFIO TPH virtualization patches to uniformly query basic TPH
existence and Extended TPH capability.

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index ea777cc33ae61..c080cf5401a31 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c

[ ... ]

> @@ -596,3 +596,22 @@ void pci_tph_init(struct pci_dev *pdev)
>  	save_size = sizeof(u32) + num_entries * sizeof(u16);
>  	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);
>  }
> +
> +/**
> + * pcie_tph_supported - Check TPH capability attribute
> + * @pdev: PCI device to query
> + * @want_ext: false - check TPH cap exists; true - check EXT_TPH support
> + *
> + * Return: true on matched condition, false otherwise
> + */
> +bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext)
> +{
> +	if (!pdev->tph_cap)
> +		return false;

[Severity: Medium]
Does this introduce a regression by failing to check the system-wide
pci_tph_disabled flag?

Since pci_tph_disabled is static to drivers/pci/tph.c, external callers like
VFIO have no other way to know if TPH has been administratively disabled. If
the host is booted with the 'notph' kernel parameter, this will still return
true.

If this incorrectly advertises the TPH capability to a guest VM via VFIO,
could it lead to an inconsistent state when the guest attempts to configure it,
since other functions honor the pci_tph_disabled flag?

> +
> +	if (!want_ext)
> +		return true;
> +
> +	return pdev->tph_ext_support;
> +}
> +EXPORT_SYMBOL(pcie_tph_supported);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=7

Re: [PATCH v19 03/18] PCI/TPH: Cache TPH requester capability at probe time

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Memory leak of `idx_data` in `mlx5_st_dealloc_index` when refcount drops to zero.
--

commit 2d469f948c634c33afd2be55f065a0101647132f
Author: Chengwen Feng <fengchengwen@huawei.com>

PCI/TPH: Cache TPH requester capability at probe time

Calculate the negotiated TPH requester type from device and root port
capabilities once in pci_tph_init().

> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
> index 489487158b99e..4097787ea98a1 100644
> --- a/drivers/pci/tph.c
> +++ b/drivers/pci/tph.c

[ ... ]

[Severity: High]
This is a pre-existing issue, but does mlx5_st_dealloc_index() leak memory
when the reference count drops to zero?

drivers/net/ethernet/mellanox/mlx5/core/lib/st.c:mlx5_st_dealloc_index() {
...
	if (refcount_dec_and_test(&idx_data->usecount)) {
		xa_erase(&st->idx_xa, st_index);
		/* We leave PCI config space as was before, no mkey will refer to it */
	}
...
}

xa_erase() removes the entry from the XArray and returns its pointer, but the
returned pointer is ignored and the idx_data object is never freed. Can this
accumulate leaked memory over time with repeated steering tag allocations?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=3

Re: [PATCH v19 08/18] PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM

[sashiko-bot]

> Add new helper pci_tph_dsm_supported() to check whether the root port
> of a given PCI device exposes the TPH ST ACPI _DSM method.
> 
> Wrap all ACPI logic under CONFIG_ACPI guard, return false directly
> when ACPI is disabled.
> 
> This helper is prepared for follow-up VFIO TPH virtualization patches
> to uniformly judge platform CPU TPH resolving support.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=8

Re: [PATCH v19 12/18] vfio/pci: Add dmabuf TPH metadata storage and fd query helper

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] vfio_pci_dma_buf_get_tph_by_fd() fails to acquire the dma_resv lock before calling vfio_pci_dma_buf_get_pci_tph(), leading to a guaranteed lockdep splat and a potential data race.
--

commit cb20e399309e26c4cb897d3a87f672443b48caa4
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Add dmabuf TPH metadata storage and fd query helper

This commit adds TPH metadata fields to vfio_pci_dma_buf to store
PH and ST information along with their validity flags. It also implements
helpers to fetch this metadata from a dmabuf, providing the foundation
for upcoming TPH resolution features.

> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
> index c16f460c01d68..d9c5e434e586f 100644
> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
[ ... ]
> @@ -81,6 +88,26 @@ static void vfio_pci_dma_buf_unmap(struct dma_buf_attachment *attachment,
>  	kref_put(&priv->kref, vfio_pci_dma_buf_done);
>  }
>  
> +static int vfio_pci_dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
> +					u16 *tag, u8 *ph)
> +{
> +	struct vfio_pci_dma_buf *priv = dmabuf->priv;
> +
> +	dma_resv_assert_held(priv->dmabuf->resv);
> +
> +	if (extended) {
[ ... ]
> @@ -106,6 +133,26 @@ static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
>  	.release = vfio_pci_dma_buf_release,
>  };
>  
> +int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended, u16 *tag, u8 *ph)
> +{
> +	struct dma_buf *dmabuf;
> +	int ret = 0;
> +
> +	dmabuf = dma_buf_get(fd);
> +	if (IS_ERR(dmabuf))
> +		return PTR_ERR(dmabuf);
> +
> +	if (dmabuf->ops != &vfio_pci_dmabuf_ops) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);

[Severity: High]
Is it necessary to acquire the dmabuf->resv lock before making this call?

Since vfio_pci_dma_buf_get_pci_tph() asserts the lock is held, it seems
invoking it directly from here will guarantee a lockdep splat.

It also appears this could leave the read of the TPH metadata bitfields
unprotected against concurrent updates, resulting in a data race.

> +out:
> +	dma_buf_put(dmabuf);
> +	return ret;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=12

Re: [PATCH v19 05/18] PCI/TPH: Refactor pcie_tph_get_cpu_st & add explicit variant

[sashiko-bot]

> Refactor pcie_tph_get_cpu_st(): extract core logic into static internal
> get_cpu_st() helper accepting explicit requester type parameter.
> 
> - Preserve original pcie_tph_get_cpu_st() unchanged as auto wrapper; it
>   uses existing pdev->tph_req_type automatically, existing callers require
>   no change.
> - Add exported pcie_tph_get_cpu_st_explicit() with bool 'extended'
>   parameter for manual STD/EXT requester selection, consumed by upcoming
>   VFIO TPH code.
> - Add capability check: reject explicit EXT request when device does not
>   support extended TPH requester.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=5

Re: [PATCH v19 18/18] vfio/pci: Expose tph_policy via debugfs

[sashiko-bot]

> Add debugfs read-only file "tph_policy" to report per-device TPH policy
> value.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=18

Re: [PATCH v19 17/18] vfio/pci: Reset hardware TPH state on device enable/disable

[sashiko-bot]

> VFIO allows user space to enable and modify PCI TPH state, but device
> reset only clears hardware TPH registers, while the software TPH state
> maintained in pci/tph.c remains stale.
> 
> Fix the stale state leakage by explicitly disabling TPH in core enable
> and disable paths:
> - In vfio_pci_core_enable(), clear leftover TPH state from previous
>   user sessions before device initialization.
> - In vfio_pci_core_disable(), revert any user-modified TPH state
>   when releasing the device.
> 
> This ensures consistent TPH hardware and software state for each
> user session and avoids residual state contamination.
> 
> Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=17

Re: [PATCH v19 09/18] vfio/pci: Hide TPH capability when TPH is unsupported

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Hiding the TPH capability by returning a length of 0 unintentionally grants the guest raw read/write access to the physical TPH registers.
--

commit 1d6f16c85a6a295f7dee447fc8d7ac56c232977a
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Hide TPH capability when TPH is unsupported

Check the device negotiated TPH support status before parsing the TPH
extended capability. Return zero length to hide the capability from
userspace if TPH is disabled during topology negotiation.

> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
> index a10ed733f0e3a..5c6ab172df6c0 100644
> --- a/drivers/vfio/pci/vfio_pci_config.c
> +++ b/drivers/vfio/pci/vfio_pci_config.c
[ ... ]
> @@ -1450,6 +1451,8 @@ static int vfio_ext_cap_len(struct vfio_pci_core_device *vdev, u16 ecap, u16 epo
>  		byte &= PCI_DPA_CAP_SUBSTATE_MASK;
>  		return PCI_DPA_BASE_SIZEOF + byte + 1;
>  	case PCI_EXT_CAP_ID_TPH:
> +		if (!pcie_tph_supported(pdev, false))
> +			return 0;

[Severity: High]
Does hiding the capability by returning 0 here inadvertently grant the guest
raw read/write access to the physical TPH registers?

When a length of 0 is returned in vfio_ext_cap_len(), the VFIO capability
parsing loop skips mapping the capability. This leaves the physical offsets
marked as PCI_CAP_ID_INVALID in vdev->pci_config_map.

Guest accesses to unmapped offsets fall back to unassigned_perms:

static struct perm_bits unassigned_perms = {
	.readfn = vfio_raw_config_read,
	.writefn = vfio_raw_config_write
};

This allows a guest to perform raw writes directly to the physical device
using vfio_raw_config_write(), bypassing VFIO isolation.

Could a malicious guest use this raw access to manually enable TPH and
configure Steering Tags to steer DMA to arbitrary CPU caches?

>  		ret = pci_read_config_dword(pdev, epos + PCI_TPH_CAP, &dword);
>  		if (ret)
>  			return pcibios_err_to_errno(ret);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=9

Re: [PATCH v19 15/18] vfio/pci: Implement TPH_RESOLVE feature for DMABUF and CPU source resolving

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [High] Flawed bitwise logic in flag validation allows userspace to bypass TPH policy restrictions and breaks forward UAPI compatibility.

Pre-existing issues:
- [High] Missing reservation lock `dma_resv_lock` before calling `vfio_pci_dma_buf_get_pci_tph` leads to a data race and lockdep assertion failure.
- [Medium] The VFIO_DEVICE_FEATURE_TPH SET operation fails to validate that the provided payload flags are zero, breaking forward compatibility.
--

commit 0733cb5339faa744645461c3f2055477418d1f5e
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Implement TPH_RESOLVE feature for DMABUF and CPU source resolving

This commit adds vfio_pci_core_feature_tph_resolve() to implement the
VFIO_DEVICE_FEATURE_TPH_RESOLVE GET-only uAPI. It resolves PH/ST values 
from two supported sources: DMABUF fds and CPU IDs, dynamically gating 
source types based on the tph_policy level.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 14944d3ea86e6..ed6bbffa8b26a 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -1641,6 +1641,78 @@ static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
>  	return copy_to_user(arg, &tph, sizeof(tph)) ? -EFAULT : 0;
>  }

[Severity: Medium]
This is a pre-existing issue in vfio_pci_core_feature_tph() just above this
new code, but does the SET operation properly validate the user payload?

When VFIO_DEVICE_FEATURE_SET is passed, the function exits early without
reading the user's struct vfio_device_feature_tph payload via 
copy_from_user():

vfio_pci_core_feature_tph() {
    if (flags & VFIO_DEVICE_FEATURE_SET) {
        vdev->tph_permit = 1;
        return 0;
    }
}

Does this silently ignore any non-zero flags in the payload, potentially
breaking UAPI forward compatibility if new flags are added later?

> +static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
> +			u32 flags,
> +			struct vfio_device_feature_tph_resolve __user *arg,
> +			size_t argsz)
> +{

[ ... ]

> +	if (copy_from_user(&resolve, arg, sizeof(resolve)))
> +		return -EFAULT;
> +
> +	src_bits = resolve.flags & VFIO_DEVICE_TPH_SRC_MASK;
> +	if (!(resolve.flags & permit_flags) || !is_power_of_2(src_bits))
> +		return -EINVAL;

[Severity: High]
Does this validation logic correctly reject unsupported flags?

Since permit_flags unconditionally includes VFIO_DEVICE_TPH_EXTENDED,
wouldn't any user payload that sets VFIO_DEVICE_TPH_EXTENDED along with
an unpermitted flag bypass this check entirely? The bitwise AND would be
non-zero, making the NOT condition false and bypassing the policy rejection.

Could this be updated to use a strict subset check instead? For example:
    if ((resolve.flags & ~permit_flags) || !is_power_of_2(src_bits))

> +	extended = !!(resolve.flags & VFIO_DEVICE_TPH_EXTENDED);
> +	if (extended && !pcie_tph_supported(vdev->pdev, true))
> +		return -EINVAL;
> +
> +	resolve.valid = 0;
> +	resolve.ph = 0;
> +	resolve.st = 0;
> +
> +	if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF) {
> +		ret = vfio_pci_dma_buf_get_tph_by_fd(resolve.src, extended,
> +						     &tag, &ph);

[Severity: High]
This isn't a bug introduced by this patch, but does calling 
vfio_pci_dma_buf_get_tph_by_fd() here trigger a lockdep splat?

Looking at vfio_pci_dma_buf_get_tph_by_fd() in
drivers/vfio/pci/vfio_pci_dmabuf.c, it delegates directly to
vfio_pci_dma_buf_get_pci_tph() without acquiring the dma_resv_lock:

vfio_pci_dma_buf_get_tph_by_fd() {
    dmabuf = dma_buf_get(fd);
    ...
    ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);
}

However, vfio_pci_dma_buf_get_pci_tph() expects the reservation lock
to be held, and asserts it:

vfio_pci_dma_buf_get_pci_tph() {
    dma_resv_assert_held(priv->dmabuf->resv);
}

Could this lead to unprotected concurrent reads of the priv struct fields
and a data race if the lock is not taken?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=15

Re: [PATCH v19 13/18] vfio/pci: Introduce VFIO_DEVICE_FEATURE_TPH family uapi for PCI TPH control

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Missing dma_resv lock around `vfio_pci_dma_buf_get_pci_tph` in `vfio_pci_dma_buf_get_tph_by_fd`.
--

commit dbd8da4bdde63982d3f30b59ace151720f7b743f
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Introduce VFIO_DEVICE_FEATURE_TPH family uapi for PCI TPH control

Add user API definitions for PCI TPH device feature set, including three
feature IDs for global enable, resolving PH/ST, and batch programming.

> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
> index c16f460c01d68..d9c5e434e586f 100644
> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
> @@ -81,6 +88,26 @@ static void vfio_pci_dma_buf_unmap(struct dma_buf_attachment *attachment,
>  	kref_put(&priv->kref, vfio_pci_dma_buf_done);
>  }
>  
> +static int vfio_pci_dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
> +					u16 *tag, u8 *ph)
> +{
> +	struct vfio_pci_dma_buf *priv = dmabuf->priv;
> +
> +	dma_resv_assert_held(priv->dmabuf->resv);
> +
> +	if (extended) {

[ ... ]

> @@ -106,6 +133,26 @@ static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
>  	.release = vfio_pci_dma_buf_release,
>  };
>  
> +int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended, u16 *tag, u8 *ph)
> +{
> +	struct dma_buf *dmabuf;
> +	int ret = 0;
> +
> +	dmabuf = dma_buf_get(fd);
> +	if (IS_ERR(dmabuf))
> +		return PTR_ERR(dmabuf);
> +
> +	if (dmabuf->ops != &vfio_pci_dmabuf_ops) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);
              ^^^^

[Severity: High]
Is it necessary to acquire the dmabuf->resv lock before calling
vfio_pci_dma_buf_get_pci_tph()?

Since vfio_pci_dma_buf_get_pci_tph() has an explicit dma_resv_assert_held(),
calling it here without the lock could trigger a lockdep assertion failure
and cause data races when reading the priv fields.

Should this code take the dma_resv_lock(dmabuf->resv, NULL) before calling
the inner getter to satisfy the assertion and prevent races?

> +out:
> +	dma_buf_put(dmabuf);
> +	return ret;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=13

Re: [PATCH v19 16/18] vfio/pci: Implement TPH_ST feature for batch ST table programming

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] TPH policy bypass due to flawed bitwise validation of uAPI flags.
- [High] Error codes during batch programming are overwritten, converting failures into silent successes.
- [High] The ioctl bypasses the TPH policy restrictions for IV_ST, allowing userspace to program CPU ST tags when forbidden.
- [High] Programming MSI-X TPH entries without ensuring PCI Memory Space Enable (MSE) is active can cause fatal Machine Check Exceptions (MCE).
- [Medium] Missing strict rejection of undefined/reserved bits in uAPI flags.
- [Medium] Missing dma_resv_lock in vfio_pci_dma_buf_get_tph_by_fd() leads to unprotected access to dmabuf state.
--

commit 7b7176f5f74da5cf5059e3ca3ebda10567316f68
Author: Chengwen Feng <fengchengwen@huawei.com>
Subject: vfio/pci: Implement TPH_ST feature for batch ST table programming

This commit adds vfio_pci_core_feature_tph_st() for batch programming
of TPH ST table entries. It introduces helpers for calculating ST table
size and resolving tags. The ioctl iterates over user-provided handles
to look up and write tags while holding the memory lock and keeping the
device in D0.

> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index ed6bbffa8b26a..7251e251824a8 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -1713,6 +1713,123 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
>  	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
>  }
>  
[ ... ]
> +static int tph_get_st_tag(struct pci_dev *pdev, u32 src_bits, u32 src_hndl,
> +			  bool extended, u16 *tag)
> +{
> +	int ret = 0;
> +	u8 ph;
> +
> +	if (src_bits & VFIO_DEVICE_TPH_SRC_NONE)
> +		*tag = 0;
> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF)
> +		ret = vfio_pci_dma_buf_get_tph_by_fd(src_hndl, extended,
> +						     tag, &ph);

[Severity: Medium]
Does this path safely access the dmabuf state?

Looking at the callee:
vfio_pci_dma_buf_get_tph_by_fd()
   vfio_pci_dma_buf_get_pci_tph(dmabuf, ...)

It appears the dmabuf is accessed without acquiring its reservation
lock (dmabuf->resv). Could this lead to data races when accessing
the TPH metadata if it changes concurrently?

> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_VOLATILE)
[ ... ]
> +static int vfio_pci_core_feature_tph_st(struct vfio_pci_core_device *vdev,
> +				u32 flags,
> +				struct vfio_device_feature_tph_st __user *arg,
> +				size_t argsz)
> +{
> +	u32 permit_flags = VFIO_DEVICE_TPH_SRC_MASK | VFIO_DEVICE_TPH_EXTENDED;
> +	struct vfio_device_feature_tph_st tph_st = {0};
> +	struct pci_dev *pdev = vdev->pdev;
> +	u32 src_bits, st_size;
> +	bool stop_on_zero_st;
> +	u32 *src_hndl = NULL;
> +	void __user *uptr;
> +	bool extended;
> +	int ret, i;
> +	u16 tag;
> +
> +	if (!vdev->tph_permit || vdev->tph_policy == VFIO_PCI_TPH_POLICY_NO_ST)
> +		return -EOPNOTSUPP;
> +
> +	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET,
> +				 sizeof(tph_st));
> +	if (ret <= 0)
> +		return ret;
> +
> +	if (copy_from_user(&tph_st, arg, sizeof(tph_st)))
> +		return -EFAULT;
> +
> +	src_bits = tph_st.flags & VFIO_DEVICE_TPH_SRC_MASK;
> +	if (vdev->tph_policy != VFIO_PCI_TPH_POLICY_LITERAL)
> +		permit_flags &= ~VFIO_DEVICE_TPH_SRC_LITERAL;

[Severity: High]
Is this missing a restriction for the VFIO_PCI_TPH_POLICY_IV_ST
policy?

In vfio_pci_core_feature_tph_resolve(), the IV_ST policy strictly
limits allowed sources to VFIO_DEVICE_TPH_SRC_DMABUF. However, here
permit_flags is initialized to VFIO_DEVICE_TPH_SRC_MASK and only
LITERAL is explicitly removed. Could this allow userspace to program
CPU ST tags when the device policy forbids it?

> +	if (!(tph_st.flags & permit_flags) || !is_power_of_2(src_bits))
> +		return -EINVAL;

[Severity: High]
Can this bitwise validation be bypassed if a user sets a forbidden
source flag along with a permitted flag?

If VFIO_DEVICE_TPH_EXTENDED is included in permit_flags and
provided in tph_st.flags, the bitwise AND evaluates to non-zero,
satisfying the first condition. If only one forbidden source bit is
set, is_power_of_2() also passes, completely bypassing the policy
restrictions.

(Note: This same validation pattern appears to be present in
vfio_pci_core_feature_tph_resolve() as well).

[Severity: Medium]
Does this logic ignore undefined or reserved bits in tph_st.flags
instead of strictly rejecting them?

If userspace passes uninitialized garbage in the upper bits, the kernel
won't return -EINVAL, potentially breaking forward compatibility for
future extensions.

> +	extended = !!(tph_st.flags & VFIO_DEVICE_TPH_EXTENDED);
[ ... ]
> +	for (i = 0; i < tph_st.count; i++) {
> +		ret = tph_get_st_tag(pdev, src_bits, src_hndl ? src_hndl[i] : 0,
> +				     extended, &tag);
> +		if (ret || (stop_on_zero_st && tag == 0))
> +			break;
> +		ret = pcie_tph_set_st_entry(pdev, tph_st.start + i, tag);

[Severity: High]
Is it safe to program MSI-X TPH entries without ensuring the PCI
Memory Space Enable (MSE) bit is active?

While this function ensures the device is in the D0 power state,
userspace can dynamically disable PCI_COMMAND_MEMORY. Since
pcie_tph_set_st_entry() performs direct MMIO writes to the BAR,
could executing this unconditionally on a disabled BAR result in an
Unsupported Request (UR) and a fatal Machine Check Exception (MCE)?

> +		if (ret)
> +			break;
> +	}
> +	ret = i;

[Severity: High]
Does this unconditional assignment destroy the error code if the
loop breaks early?

If tph_get_st_tag() or pcie_tph_set_st_entry() fails on the very
first iteration, ret will hold a negative error code (like
-EINVAL). When the loop breaks, ret = i; overwrites it with 0,
causing the ioctl to silently return success to userspace.

> +
> +out:
> +	up_write(&vdev->memory_lock);
> +	kfree(src_hndl);
> +	return ret;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=16

Re: [PATCH v19 11/18] vfio/pci: Virtualize PCIe TPH capability registers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Critical] Out-of-bounds read and write on the `perm->virt` and `perm->write` kernel heap buffers due to unvalidated dynamic hardware TPH capability length.

Pre-existing issues:
- [Critical] Missing bounds check in `vfio_ecap_init` can lead to a heap buffer overflow when handling extended capabilities with dynamically parsed lengths.
--

commit 78dd911822344404c5af18d9dd92fb51e5d34a18
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Virtualize PCIe TPH capability registers

This patch implements virtualization and policy masking for the PCIe TPH
extended capability. It splits configuration space permissions to keep the
header read-only while marking TPH_CTRL and ST-table entries as virtually
writable, and adjusts virtual capability bits based on hardware support.

> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
> index 5c6ab172df6c0..06d7b2fbf866a 100644
> --- a/drivers/vfio/pci/vfio_pci_config.c
> +++ b/drivers/vfio/pci/vfio_pci_config.c
> @@ -1086,6 +1086,73 @@ static int __init init_pci_ext_cap_pwr_perm(struct perm_bits *perm)
>  	return 0;
>  }
>  
> +/* Permissions for TPH extended capability */
> +static int __init init_pci_ext_cap_tph_perm(struct perm_bits *perm)
> +{
> +	int i;
> +
> +	if (alloc_perm_bits(perm, pci_ext_cap_length[PCI_EXT_CAP_ID_TPH]))

[Severity: Critical]
Does this code under-allocate the permission bits if the capability length
is dynamically larger than 256 bytes?

Because pci_ext_cap_length[PCI_EXT_CAP_ID_TPH] is 0xFF, alloc_perm_bits()
will allocate a fixed 256-byte buffer. If vfio_ext_cap_len() calculates a
dynamically larger TPH capability size based on a device's ST table, it
appears subsequent user space reads or writes past offset 255 could cause an
out-of-bounds heap access when vfio_default_config_read() or
vfio_default_config_write() calls memcpy() against perm->virt and perm->write.

> +		return -ENOMEM;
> +
> +	p_setd(perm, 0, ALL_VIRT, NO_WRITE);

[ ... ]

> @@ -1704,6 +1774,8 @@ static int vfio_ecap_init(struct vfio_pci_core_device *vdev)
>  		ret = vfio_fill_vconfig_bytes(vdev, epos, len);
>  		if (ret)
>  			return ret;
> +		if (ecap == PCI_EXT_CAP_ID_TPH && !hidden)
> +			vfio_tph_capability_adjust(vdev, epos);

[Severity: Critical]
This is a pre-existing issue, but could the dynamically calculated length lead
to a heap buffer overflow here?

If a faulty or malicious device advertises a large ST table size,
vfio_ext_cap_len() can calculate a length where epos + len significantly
exceeds pdev->cfg_size. Since there appears to be no bounds check before:

drivers/vfio/pci/vfio_pci_config.c:vfio_ecap_init() {
    ...
    memset(map + epos, ecap, len);
    ret = vfio_fill_vconfig_bytes(vdev, epos, len);
    ...
}

could this overflow the vdev->pci_config_map and vdev->vconfig heap buffers?

>  
>  		/*
>  		 * If we're just using this capability to anchor the list,

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=11

Re: [PATCH v19 14/18] vfio/pci: Implement VFIO_DEVICE_FEATURE_TPH and valid TPH config write support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [High] Waking the device to PCI_D0 during TPH configuration writes or ST table programming fails to restore zapped DMA buffers, permanently breaking device DMA.
- [Medium] The `VFIO_DEVICE_FEATURE_SET` operation for TPH fails to copy and validate the userspace payload, breaking forward compatibility.

Pre-existing issues:
- [High] Missing bounds checking in PCI capability parsing loops allows a malicious or spoofed PCIe endpoint to overflow the `vconfig` and `pci_config_map` heap buffers.
--

commit 6662e41b04cb5eb843722e0409b68031565e33d6
Author: Chengwen Feng <fengchengwen@huawei.com>

vfio/pci: Implement VFIO_DEVICE_FEATURE_TPH and valid TPH config write support

This commit adds support for VFIO_DEVICE_FEATURE_TPH in
vfio_pci_core_feature_tph, allowing GET and SET operations for TPH features.
It also implements vfio_tph_config_write to support TPH_CTRL modifications once
TPH is permitted, managing power states as required for writing TPH registers.

> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
> index 06d7b2fbf866a..388dd6fed16b5 100644
> --- a/drivers/vfio/pci/vfio_pci_config.c
> +++ b/drivers/vfio/pci/vfio_pci_config.c
> @@ -1150,6 +1150,60 @@ static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
>  				 int count, struct perm_bits *perm,
>  				 int offset, __le32 val)
>  {
[ ... ]
> +	org_req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, org_ctrl);
> +	if (req == org_req)
> +		goto out; /* Only care about requester enable */
> +
> +	ret = vfio_pci_set_power_state(vdev, PCI_D0);
> +	if (ret)
> +		goto restore; /* Drop this write */

[Severity: High]
Does waking the device to PCI_D0 here miss the required vfio_pci_dma_buf_move()
call to restore DMA mappings?

If the device is in a low-power state (like D3hot) and its DMA buffers are
zapped, forcing it to D0 without restoring the DMA mappings could leave the
device active but unable to perform DMA, potentially causing IOMMU faults and
breaking device functionality. Does the same issue also occur in the ioctl path
for ST table batch programming?

[ ... ]
> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 3f11a9624b9c0..14944d3ea86e6 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
[ ... ]
> @@ -1607,6 +1609,38 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
>  	return 0;
>  }
>  
> +static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
> +				     u32 flags,
> +				     struct vfio_device_feature_tph __user *arg,
> +				     size_t argsz)
> +{
> +	struct vfio_device_feature_tph tph = {0};
> +	int ret;
> +
> +	if (!pcie_tph_supported(vdev->pdev, false))
> +		return -EOPNOTSUPP;
> +
> +	ret = vfio_check_feature(flags, argsz,
> +			VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_SET,
> +			sizeof(tph));
> +	if (ret <= 0)
> +		return ret;
> +
> +	if (flags & VFIO_DEVICE_FEATURE_SET) {
> +		vdev->tph_permit = 1;
> +		return 0;
> +	}

[Severity: Medium]
Does this code silently ignore userspace payloads for the VFIO_DEVICE_FEATURE_SET
operation by skipping copy and validation?

Future kernel extensions that add new flags to struct vfio_device_feature_tph
for the SET operation could be silently ignored by older kernels, which might
break userspace's ability to safely probe or use new features. Should it verify
that the payload is zeroed?

[ ... ]

While reviewing this code, I also noticed a pre-existing issue in
vfio_ecap_init() in drivers/vfio/pci/vfio_pci_config.c:

[Severity: High]
This is a pre-existing issue, but does the while loop in vfio_ecap_init()
lack a bounds check for the heap buffers?

drivers/vfio/pci/vfio_pci_config.c:vfio_ecap_init() {
[ ... ]
	while (loops-- && epos >= PCI_CFG_SPACE_SIZE) {
[ ... ]
		for (i = 0; i < len; i++) {
			if (likely(map[epos + i] == PCI_CAP_ID_INVALID))
				continue;
[ ... ]
		memset(map + epos, ecap, len);
		ret = vfio_fill_vconfig_bytes(vdev, epos, len);
[ ... ]
}

The loop condition only checks that epos >= PCI_CFG_SPACE_SIZE, but never
verifies that epos + len <= pdev->cfg_size before accessing the heap
allocations (map and vconfig). Could a malicious or spoofed PCIe endpoint
provide an ecap with an epos and len that overflows these buffers?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260702124224.57168-1-fengchengwen@huawei.com?part=14

Re: [PATCH v19 09/18] vfio/pci: Hide TPH capability when TPH is unsupported

[fengchengwen]

On 7/2/2026 9:00 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Hiding the TPH capability by returning a length of 0 unintentionally grants the guest raw read/write access to the physical TPH registers.

After checking the full vfio_ecap_init logic, this is a false positive risk
report.

When vfio_ext_cap_len() returns 0 for unsupported TPH ecap, the !len branch
takes over:
1. It allocates minimal PCI_CAP_SIZEOF length and marks hidden=true;
2. memset(map + epos, PCI_EXT_CAP_ID_TPH, len) populates pci_config_map,
   so these offsets are never marked PCI_CAP_ID_INVALID;
3. Accesses to TPH range hit the dedicated TPH ecap permission table
   with vfio_tph_config_write handler which discards all writes,
   never falling back to unassigned_perms raw physical write.

The hidden flag erases the capability ID in virtual config space so guests
cannot enumerate TPH, while full permission isolation is retained for all
config offsets of this ecap. No raw hardware bypass vulnerability exists
with current implementation.

Thanks

> --
> 
> commit 1d6f16c85a6a295f7dee447fc8d7ac56c232977a
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> vfio/pci: Hide TPH capability when TPH is unsupported
> 
> Check the device negotiated TPH support status before parsing the TPH
> extended capability. Return zero length to hide the capability from
> userspace if TPH is disabled during topology negotiation.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
>> index a10ed733f0e3a..5c6ab172df6c0 100644
>> --- a/drivers/vfio/pci/vfio_pci_config.c
>> +++ b/drivers/vfio/pci/vfio_pci_config.c
> [ ... ]
>> @@ -1450,6 +1451,8 @@ static int vfio_ext_cap_len(struct vfio_pci_core_device *vdev, u16 ecap, u16 epo
>>  		byte &= PCI_DPA_CAP_SUBSTATE_MASK;
>>  		return PCI_DPA_BASE_SIZEOF + byte + 1;
>>  	case PCI_EXT_CAP_ID_TPH:
>> +		if (!pcie_tph_supported(pdev, false))
>> +			return 0;
> 
> [Severity: High]
> Does hiding the capability by returning 0 here inadvertently grant the guest
> raw read/write access to the physical TPH registers?
> 
> When a length of 0 is returned in vfio_ext_cap_len(), the VFIO capability
> parsing loop skips mapping the capability. This leaves the physical offsets
> marked as PCI_CAP_ID_INVALID in vdev->pci_config_map.
> 
> Guest accesses to unmapped offsets fall back to unassigned_perms:
> 
> static struct perm_bits unassigned_perms = {
> 	.readfn = vfio_raw_config_read,
> 	.writefn = vfio_raw_config_write
> };
> 
> This allows a guest to perform raw writes directly to the physical device
> using vfio_raw_config_write(), bypassing VFIO isolation.
> 
> Could a malicious guest use this raw access to manually enable TPH and
> configure Steering Tags to steer DMA to arbitrary CPU caches?
> 
>>  		ret = pci_read_config_dword(pdev, epos + PCI_TPH_CAP, &dword);
>>  		if (ret)
>>  			return pcibios_err_to_errno(ret);
> 

Re: [PATCH v19 07/18] PCI/TPH: Add pcie_tph_supported() helper to check TPH capability attributes

[fengchengwen]

On 7/2/2026 8:53 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] pcie_tph_supported() does not check the system-wide pci_tph_disabled flag, leading to falsely reporting TPH support when it has been explicitly disabled by the administrator.

Thanks for the review note. I'd prefer not to add pci_tph_disabled inside
pcie_tph_supported() for the following reasons:

- The helper's sole purpose is to report hardware/topology TPH capability,
  while pci_tph_disabled is an administrative gate only applied at TPH activation
  entry points (pcie_enable_tph / pcie_enable_tph_explicit). All attempts to
  turn on TPH are already blocked when notph is set, so there is no bypass risk.
- Adding the global flag here would require replicating the check across all
  TPH query helpers for consistency, which adds redundant overhead.
- Distinguishing hardware presence vs admin disable state is useful visibility
  for userspace; hiding the capability entirely removes this distinction.

All TPH activation paths already enforce the global disable, so the reporting
mismatch does not create functional or security issues. Let me know if others
disagree with this separation of semantics.

Thanks

> --
> 
> commit 089d399dcdf1ae5d6be4d557f772bf15c0d7d461
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> PCI/TPH: Add pcie_tph_supported() helper to check TPH capability attributes
> 
> Add new helper pcie_tph_supported() to check if a device has a valid TPH
> capability or hardware Extended TPH support. This helper is intended for
> follow-up VFIO TPH virtualization patches to uniformly query basic TPH
> existence and Extended TPH capability.
> 
>> diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c
>> index ea777cc33ae61..c080cf5401a31 100644
>> --- a/drivers/pci/tph.c
>> +++ b/drivers/pci/tph.c
> 
> [ ... ]
> 
>> @@ -596,3 +596,22 @@ void pci_tph_init(struct pci_dev *pdev)
>>  	save_size = sizeof(u32) + num_entries * sizeof(u16);
>>  	pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_TPH, save_size);
>>  }
>> +
>> +/**
>> + * pcie_tph_supported - Check TPH capability attribute
>> + * @pdev: PCI device to query
>> + * @want_ext: false - check TPH cap exists; true - check EXT_TPH support
>> + *
>> + * Return: true on matched condition, false otherwise
>> + */
>> +bool pcie_tph_supported(struct pci_dev *pdev, bool want_ext)
>> +{
>> +	if (!pdev->tph_cap)
>> +		return false;
> 
> [Severity: Medium]
> Does this introduce a regression by failing to check the system-wide
> pci_tph_disabled flag?
> 
> Since pci_tph_disabled is static to drivers/pci/tph.c, external callers like
> VFIO have no other way to know if TPH has been administratively disabled. If
> the host is booted with the 'notph' kernel parameter, this will still return
> true.
> 
> If this incorrectly advertises the TPH capability to a guest VM via VFIO,
> could it lead to an inconsistent state when the guest attempts to configure it,
> since other functions honor the pci_tph_disabled flag?
> 
>> +
>> +	if (!want_ext)
>> +		return true;
>> +
>> +	return pdev->tph_ext_support;
>> +}
>> +EXPORT_SYMBOL(pcie_tph_supported);
> 

Re: [PATCH v19 11/18] vfio/pci: Virtualize PCIe TPH capability registers

[fengchengwen]

Thanks for the review.

On 7/2/2026 9:04 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> 
> New issues:
> - [Critical] Out-of-bounds read and write on the `perm->virt` and `perm->write` kernel heap buffers due to unvalidated dynamic hardware TPH capability length.

TPH perm buffer OOB: Safe per PCI spec, ST table capped at 64 entries,
all permission setup stays under 0xFF limit, VFIO access validates perm
ranges.

> 
> Pre-existing issues:
> - [Critical] Missing bounds check in `vfio_ecap_init` can lead to a heap buffer overflow when handling extended capabilities with dynamically parsed lengths.

vfio_ecap_init overflow: Pre-existing generic issue for all dynamic ecaps,
not introduced by this patch; I think we need another patch for this (not
in this serial).

Thanks

> --
> 
> commit 78dd911822344404c5af18d9dd92fb51e5d34a18
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> vfio/pci: Virtualize PCIe TPH capability registers
> 
> This patch implements virtualization and policy masking for the PCIe TPH
> extended capability. It splits configuration space permissions to keep the
> header read-only while marking TPH_CTRL and ST-table entries as virtually
> writable, and adjusts virtual capability bits based on hardware support.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
>> index 5c6ab172df6c0..06d7b2fbf866a 100644
>> --- a/drivers/vfio/pci/vfio_pci_config.c
>> +++ b/drivers/vfio/pci/vfio_pci_config.c
>> @@ -1086,6 +1086,73 @@ static int __init init_pci_ext_cap_pwr_perm(struct perm_bits *perm)
>>  	return 0;
>>  }
>>  
>> +/* Permissions for TPH extended capability */
>> +static int __init init_pci_ext_cap_tph_perm(struct perm_bits *perm)
>> +{
>> +	int i;
>> +
>> +	if (alloc_perm_bits(perm, pci_ext_cap_length[PCI_EXT_CAP_ID_TPH]))
> 
> [Severity: Critical]
> Does this code under-allocate the permission bits if the capability length
> is dynamically larger than 256 bytes?
> 
> Because pci_ext_cap_length[PCI_EXT_CAP_ID_TPH] is 0xFF, alloc_perm_bits()
> will allocate a fixed 256-byte buffer. If vfio_ext_cap_len() calculates a
> dynamically larger TPH capability size based on a device's ST table, it
> appears subsequent user space reads or writes past offset 255 could cause an
> out-of-bounds heap access when vfio_default_config_read() or
> vfio_default_config_write() calls memcpy() against perm->virt and perm->write.
> 
>> +		return -ENOMEM;
>> +
>> +	p_setd(perm, 0, ALL_VIRT, NO_WRITE);
> 
> [ ... ]
> 
>> @@ -1704,6 +1774,8 @@ static int vfio_ecap_init(struct vfio_pci_core_device *vdev)
>>  		ret = vfio_fill_vconfig_bytes(vdev, epos, len);
>>  		if (ret)
>>  			return ret;
>> +		if (ecap == PCI_EXT_CAP_ID_TPH && !hidden)
>> +			vfio_tph_capability_adjust(vdev, epos);
> 
> [Severity: Critical]
> This is a pre-existing issue, but could the dynamically calculated length lead
> to a heap buffer overflow here?
> 
> If a faulty or malicious device advertises a large ST table size,
> vfio_ext_cap_len() can calculate a length where epos + len significantly
> exceeds pdev->cfg_size. Since there appears to be no bounds check before:
> 
> drivers/vfio/pci/vfio_pci_config.c:vfio_ecap_init() {
>     ...
>     memset(map + epos, ecap, len);
>     ret = vfio_fill_vconfig_bytes(vdev, epos, len);
>     ...
> }
> 
> could this overflow the vdev->pci_config_map and vdev->vconfig heap buffers?
> 
>>  
>>  		/*
>>  		 * If we're just using this capability to anchor the list,
> 

Re: [PATCH v19 12/18] vfio/pci: Add dmabuf TPH metadata storage and fd query helper

[fengchengwen]

On 7/2/2026 8:56 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] vfio_pci_dma_buf_get_tph_by_fd() fails to acquire the dma_resv lock before calling vfio_pci_dma_buf_get_pci_tph(), leading to a guaranteed lockdep splat and a potential data race.

Thanks for catching this critical lock omission, this is a real bug I missed.
Will fix in v20

> --
> 
> commit cb20e399309e26c4cb897d3a87f672443b48caa4
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> vfio/pci: Add dmabuf TPH metadata storage and fd query helper
> 
> This commit adds TPH metadata fields to vfio_pci_dma_buf to store
> PH and ST information along with their validity flags. It also implements
> helpers to fetch this metadata from a dmabuf, providing the foundation
> for upcoming TPH resolution features.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
>> index c16f460c01d68..d9c5e434e586f 100644
>> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
>> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
> [ ... ]
>> @@ -81,6 +88,26 @@ static void vfio_pci_dma_buf_unmap(struct dma_buf_attachment *attachment,
>>  	kref_put(&priv->kref, vfio_pci_dma_buf_done);
>>  }
>>  
>> +static int vfio_pci_dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
>> +					u16 *tag, u8 *ph)
>> +{
>> +	struct vfio_pci_dma_buf *priv = dmabuf->priv;
>> +
>> +	dma_resv_assert_held(priv->dmabuf->resv);
>> +
>> +	if (extended) {
> [ ... ]
>> @@ -106,6 +133,26 @@ static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
>>  	.release = vfio_pci_dma_buf_release,
>>  };
>>  
>> +int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended, u16 *tag, u8 *ph)
>> +{
>> +	struct dma_buf *dmabuf;
>> +	int ret = 0;
>> +
>> +	dmabuf = dma_buf_get(fd);
>> +	if (IS_ERR(dmabuf))
>> +		return PTR_ERR(dmabuf);
>> +
>> +	if (dmabuf->ops != &vfio_pci_dmabuf_ops) {
>> +		ret = -EINVAL;
>> +		goto out;
>> +	}
>> +
>> +	ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);
> 
> [Severity: High]
> Is it necessary to acquire the dmabuf->resv lock before making this call?
> 
> Since vfio_pci_dma_buf_get_pci_tph() asserts the lock is held, it seems
> invoking it directly from here will guarantee a lockdep splat.
> 
> It also appears this could leave the read of the TPH metadata bitfields
> unprotected against concurrent updates, resulting in a data race.
> 
>> +out:
>> +	dma_buf_put(dmabuf);
>> +	return ret;
>> +}
> 

Re: [PATCH v19 13/18] vfio/pci: Introduce VFIO_DEVICE_FEATURE_TPH family uapi for PCI TPH control

[fengchengwen]

On 7/2/2026 9:01 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Missing dma_resv lock around `vfio_pci_dma_buf_get_pci_tph` in `vfio_pci_dma_buf_get_tph_by_fd`.

This is a Sashiko matching error. This patch only modifies uapi vfio.h and
contains no dmabuf code. The missing dma_resv lock issue is in patch 12/18,
which I will fix separately in next version.

> --
> 
> commit dbd8da4bdde63982d3f30b59ace151720f7b743f
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> vfio/pci: Introduce VFIO_DEVICE_FEATURE_TPH family uapi for PCI TPH control
> 
> Add user API definitions for PCI TPH device feature set, including three
> feature IDs for global enable, resolving PH/ST, and batch programming.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
>> index c16f460c01d68..d9c5e434e586f 100644
>> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
>> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
>> @@ -81,6 +88,26 @@ static void vfio_pci_dma_buf_unmap(struct dma_buf_attachment *attachment,
>>  	kref_put(&priv->kref, vfio_pci_dma_buf_done);
>>  }
>>  
>> +static int vfio_pci_dma_buf_get_pci_tph(struct dma_buf *dmabuf, bool extended,
>> +					u16 *tag, u8 *ph)
>> +{
>> +	struct vfio_pci_dma_buf *priv = dmabuf->priv;
>> +
>> +	dma_resv_assert_held(priv->dmabuf->resv);
>> +
>> +	if (extended) {
> 
> [ ... ]
> 
>> @@ -106,6 +133,26 @@ static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
>>  	.release = vfio_pci_dma_buf_release,
>>  };
>>  
>> +int vfio_pci_dma_buf_get_tph_by_fd(int fd, bool extended, u16 *tag, u8 *ph)
>> +{
>> +	struct dma_buf *dmabuf;
>> +	int ret = 0;
>> +
>> +	dmabuf = dma_buf_get(fd);
>> +	if (IS_ERR(dmabuf))
>> +		return PTR_ERR(dmabuf);
>> +
>> +	if (dmabuf->ops != &vfio_pci_dmabuf_ops) {
>> +		ret = -EINVAL;
>> +		goto out;
>> +	}
>> +
>> +	ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);
>               ^^^^
> 
> [Severity: High]
> Is it necessary to acquire the dmabuf->resv lock before calling
> vfio_pci_dma_buf_get_pci_tph()?
> 
> Since vfio_pci_dma_buf_get_pci_tph() has an explicit dma_resv_assert_held(),
> calling it here without the lock could trigger a lockdep assertion failure
> and cause data races when reading the priv fields.
> 
> Should this code take the dma_resv_lock(dmabuf->resv, NULL) before calling
> the inner getter to satisfy the assertion and prevent races?
> 
>> +out:
>> +	dma_buf_put(dmabuf);
>> +	return ret;
>> +}
> 

Re: [PATCH v19 14/18] vfio/pci: Implement VFIO_DEVICE_FEATURE_TPH and valid TPH config write support

[fengchengwen]

Hi Alex,

On 7/2/2026 9:04 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> 
> New issues:
> - [High] Waking the device to PCI_D0 during TPH configuration writes or ST table programming fails to restore zapped DMA buffers, permanently breaking device DMA.

Raw vfio_pci_set_power_state() does not restore revoked dma-buf mappings, which
theoretically may trigger IOMMU faults for guests. There is a local helper
vfio_lock_and_set_power_state() that takes memory locks and runs
vfio_pci_dma_buf_move(vdev, false) after switching to D0 to rebuild DMA
IOVA mappings.

I’m considering replacing direct vfio_pci_set_power_state(PCI_D0) calls
in TPH write and TPH_ST paths with this wrapper. Could you help confirm
whether this is a valid concern and if this wrapper usage is acceptable here,
or if there are any hidden edge cases I need to handle?


> - [Medium] The `VFIO_DEVICE_FEATURE_SET` operation for TPH fails to copy and validate the userspace payload, breaking forward compatibility.

Will add copy_from_user + zero check in next version.

> 
> Pre-existing issues:
> - [High] Missing bounds checking in PCI capability parsing loops allows a malicious or spoofed PCIe endpoint to overflow the `vconfig` and `pci_config_map` heap buffers.

Pre-existing generic issue, separate cleanup patch later.

Thanks

> --
> 
> commit 6662e41b04cb5eb843722e0409b68031565e33d6
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> vfio/pci: Implement VFIO_DEVICE_FEATURE_TPH and valid TPH config write support
> 
> This commit adds support for VFIO_DEVICE_FEATURE_TPH in
> vfio_pci_core_feature_tph, allowing GET and SET operations for TPH features.
> It also implements vfio_tph_config_write to support TPH_CTRL modifications once
> TPH is permitted, managing power states as required for writing TPH registers.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
>> index 06d7b2fbf866a..388dd6fed16b5 100644
>> --- a/drivers/vfio/pci/vfio_pci_config.c
>> +++ b/drivers/vfio/pci/vfio_pci_config.c
>> @@ -1150,6 +1150,60 @@ static int vfio_tph_config_write(struct vfio_pci_core_device *vdev, int pos,
>>  				 int count, struct perm_bits *perm,
>>  				 int offset, __le32 val)
>>  {
> [ ... ]
>> +	org_req = FIELD_GET(PCI_TPH_CTRL_REQ_EN_MASK, org_ctrl);
>> +	if (req == org_req)
>> +		goto out; /* Only care about requester enable */
>> +
>> +	ret = vfio_pci_set_power_state(vdev, PCI_D0);
>> +	if (ret)
>> +		goto restore; /* Drop this write */
> 
> [Severity: High]
> Does waking the device to PCI_D0 here miss the required vfio_pci_dma_buf_move()
> call to restore DMA mappings?
> 
> If the device is in a low-power state (like D3hot) and its DMA buffers are
> zapped, forcing it to D0 without restoring the DMA mappings could leave the
> device active but unable to perform DMA, potentially causing IOMMU faults and
> breaking device functionality. Does the same issue also occur in the ioctl path
> for ST table batch programming?
> 
> [ ... ]
>> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
>> index 3f11a9624b9c0..14944d3ea86e6 100644
>> --- a/drivers/vfio/pci/vfio_pci_core.c
>> +++ b/drivers/vfio/pci/vfio_pci_core.c
> [ ... ]
>> @@ -1607,6 +1609,38 @@ static int vfio_pci_core_feature_token(struct vfio_pci_core_device *vdev,
>>  	return 0;
>>  }
>>  
>> +static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
>> +				     u32 flags,
>> +				     struct vfio_device_feature_tph __user *arg,
>> +				     size_t argsz)
>> +{
>> +	struct vfio_device_feature_tph tph = {0};
>> +	int ret;
>> +
>> +	if (!pcie_tph_supported(vdev->pdev, false))
>> +		return -EOPNOTSUPP;
>> +
>> +	ret = vfio_check_feature(flags, argsz,
>> +			VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_SET,
>> +			sizeof(tph));
>> +	if (ret <= 0)
>> +		return ret;
>> +
>> +	if (flags & VFIO_DEVICE_FEATURE_SET) {
>> +		vdev->tph_permit = 1;
>> +		return 0;
>> +	}
> 
> [Severity: Medium]
> Does this code silently ignore userspace payloads for the VFIO_DEVICE_FEATURE_SET
> operation by skipping copy and validation?
> 
> Future kernel extensions that add new flags to struct vfio_device_feature_tph
> for the SET operation could be silently ignored by older kernels, which might
> break userspace's ability to safely probe or use new features. Should it verify
> that the payload is zeroed?
> 
> [ ... ]
> 
> While reviewing this code, I also noticed a pre-existing issue in
> vfio_ecap_init() in drivers/vfio/pci/vfio_pci_config.c:
> 
> [Severity: High]
> This is a pre-existing issue, but does the while loop in vfio_ecap_init()
> lack a bounds check for the heap buffers?
> 
> drivers/vfio/pci/vfio_pci_config.c:vfio_ecap_init() {
> [ ... ]
> 	while (loops-- && epos >= PCI_CFG_SPACE_SIZE) {
> [ ... ]
> 		for (i = 0; i < len; i++) {
> 			if (likely(map[epos + i] == PCI_CAP_ID_INVALID))
> 				continue;
> [ ... ]
> 		memset(map + epos, ecap, len);
> 		ret = vfio_fill_vconfig_bytes(vdev, epos, len);
> [ ... ]
> }
> 
> The loop condition only checks that epos >= PCI_CFG_SPACE_SIZE, but never
> verifies that epos + len <= pdev->cfg_size before accessing the heap
> allocations (map and vconfig). Could a malicious or spoofed PCIe endpoint
> provide an ecap with an epos and len that overflows these buffers?
> 

Re: [PATCH v19 15/18] vfio/pci: Implement TPH_RESOLVE feature for DMABUF and CPU source resolving

[fengchengwen]

On 7/2/2026 9:00 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
> 
> New issues:
> - [High] Flawed bitwise logic in flag validation allows userspace to bypass TPH policy restrictions and breaks forward UAPI compatibility.

Real bug, will fix in v20

> 
> Pre-existing issues:
> - [High] Missing reservation lock `dma_resv_lock` before calling `vfio_pci_dma_buf_get_pci_tph` leads to a data race and lockdep assertion failure.
> - [Medium] The VFIO_DEVICE_FEATURE_TPH SET operation fails to validate that the provided payload flags are zero, breaking forward compatibility.

Pre-commits bug, will fix in v20

Thanks

> --
> 
> commit 0733cb5339faa744645461c3f2055477418d1f5e
> Author: Chengwen Feng <fengchengwen@huawei.com>
> 
> vfio/pci: Implement TPH_RESOLVE feature for DMABUF and CPU source resolving
> 
> This commit adds vfio_pci_core_feature_tph_resolve() to implement the
> VFIO_DEVICE_FEATURE_TPH_RESOLVE GET-only uAPI. It resolves PH/ST values 
> from two supported sources: DMABUF fds and CPU IDs, dynamically gating 
> source types based on the tph_policy level.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
>> index 14944d3ea86e6..ed6bbffa8b26a 100644
>> --- a/drivers/vfio/pci/vfio_pci_core.c
>> +++ b/drivers/vfio/pci/vfio_pci_core.c
>> @@ -1641,6 +1641,78 @@ static int vfio_pci_core_feature_tph(struct vfio_pci_core_device *vdev,
>>  	return copy_to_user(arg, &tph, sizeof(tph)) ? -EFAULT : 0;
>>  }
> 
> [Severity: Medium]
> This is a pre-existing issue in vfio_pci_core_feature_tph() just above this
> new code, but does the SET operation properly validate the user payload?
> 
> When VFIO_DEVICE_FEATURE_SET is passed, the function exits early without
> reading the user's struct vfio_device_feature_tph payload via 
> copy_from_user():
> 
> vfio_pci_core_feature_tph() {
>     if (flags & VFIO_DEVICE_FEATURE_SET) {
>         vdev->tph_permit = 1;
>         return 0;
>     }
> }
> 
> Does this silently ignore any non-zero flags in the payload, potentially
> breaking UAPI forward compatibility if new flags are added later?
> 
>> +static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
>> +			u32 flags,
>> +			struct vfio_device_feature_tph_resolve __user *arg,
>> +			size_t argsz)
>> +{
> 
> [ ... ]
> 
>> +	if (copy_from_user(&resolve, arg, sizeof(resolve)))
>> +		return -EFAULT;
>> +
>> +	src_bits = resolve.flags & VFIO_DEVICE_TPH_SRC_MASK;
>> +	if (!(resolve.flags & permit_flags) || !is_power_of_2(src_bits))
>> +		return -EINVAL;
> 
> [Severity: High]
> Does this validation logic correctly reject unsupported flags?
> 
> Since permit_flags unconditionally includes VFIO_DEVICE_TPH_EXTENDED,
> wouldn't any user payload that sets VFIO_DEVICE_TPH_EXTENDED along with
> an unpermitted flag bypass this check entirely? The bitwise AND would be
> non-zero, making the NOT condition false and bypassing the policy rejection.
> 
> Could this be updated to use a strict subset check instead? For example:
>     if ((resolve.flags & ~permit_flags) || !is_power_of_2(src_bits))
> 
>> +	extended = !!(resolve.flags & VFIO_DEVICE_TPH_EXTENDED);
>> +	if (extended && !pcie_tph_supported(vdev->pdev, true))
>> +		return -EINVAL;
>> +
>> +	resolve.valid = 0;
>> +	resolve.ph = 0;
>> +	resolve.st = 0;
>> +
>> +	if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF) {
>> +		ret = vfio_pci_dma_buf_get_tph_by_fd(resolve.src, extended,
>> +						     &tag, &ph);
> 
> [Severity: High]
> This isn't a bug introduced by this patch, but does calling 
> vfio_pci_dma_buf_get_tph_by_fd() here trigger a lockdep splat?
> 
> Looking at vfio_pci_dma_buf_get_tph_by_fd() in
> drivers/vfio/pci/vfio_pci_dmabuf.c, it delegates directly to
> vfio_pci_dma_buf_get_pci_tph() without acquiring the dma_resv_lock:
> 
> vfio_pci_dma_buf_get_tph_by_fd() {
>     dmabuf = dma_buf_get(fd);
>     ...
>     ret = vfio_pci_dma_buf_get_pci_tph(dmabuf, extended, tag, ph);
> }
> 
> However, vfio_pci_dma_buf_get_pci_tph() expects the reservation lock
> to be held, and asserts it:
> 
> vfio_pci_dma_buf_get_pci_tph() {
>     dma_resv_assert_held(priv->dmabuf->resv);
> }
> 
> Could this lead to unprotected concurrent reads of the priv struct fields
> and a data race if the lock is not taken?
> 
> [ ... ]
> 

Re: [PATCH v19 16/18] vfio/pci: Implement TPH_ST feature for batch ST table programming

[fengchengwen]

On 7/2/2026 9:04 PM, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
> - [High] TPH policy bypass due to flawed bitwise validation of uAPI flags.

Read bug, will fix in v20

> - [High] Error codes during batch programming are overwritten, converting failures into silent successes.

False positive because the ioctl return 0 means no entry was programed

> - [High] The ioctl bypasses the TPH policy restrictions for IV_ST, allowing userspace to program CPU ST tags when forbidden.

False positive because TPH_ST could also allowed when tph_policy=IV_ST
(allowd src: NONE/DMABUF/CPU)

> - [High] Programming MSI-X TPH entries without ensuring PCI Memory Space Enable (MSE) is active can cause fatal Machine Check Exceptions (MCE).

Two possible options to resolve this I think:
1. Add explicit MSE read/validation inside vfio_pci_core_feature_tph_st() before
   each pcie_tph_set_st_entry() call, returning -EIO when memory space is disabled.
2. Move the MSE guard into the shared pcie_tph_set_st_entry() helper in pci/tph.c,
   so all callers automatically inherit the safety check without duplicate logic.

Could reviewers advise which approach is preferred here, or confirm whether this
MSE validation is required at all for the TPH batch programming path?

> - [Medium] Missing strict rejection of undefined/reserved bits in uAPI flags.
> - [Medium] Missing dma_resv_lock in vfio_pci_dma_buf_get_tph_by_fd() leads to unprotected access to dmabuf state.

The above two are both pre-commits' bug, will fix in v20

Thanks

> --
> 
> commit 7b7176f5f74da5cf5059e3ca3ebda10567316f68
> Author: Chengwen Feng <fengchengwen@huawei.com>
> Subject: vfio/pci: Implement TPH_ST feature for batch ST table programming
> 
> This commit adds vfio_pci_core_feature_tph_st() for batch programming
> of TPH ST table entries. It introduces helpers for calculating ST table
> size and resolving tags. The ioctl iterates over user-provided handles
> to look up and write tags while holding the memory lock and keeping the
> device in D0.
> 
>> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
>> index ed6bbffa8b26a..7251e251824a8 100644
>> --- a/drivers/vfio/pci/vfio_pci_core.c
>> +++ b/drivers/vfio/pci/vfio_pci_core.c
>> @@ -1713,6 +1713,123 @@ static int vfio_pci_core_feature_tph_resolve(struct vfio_pci_core_device *vdev,
>>  	return copy_to_user(arg, &resolve, sizeof(resolve)) ? -EFAULT : 0;
>>  }
>>  
> [ ... ]
>> +static int tph_get_st_tag(struct pci_dev *pdev, u32 src_bits, u32 src_hndl,
>> +			  bool extended, u16 *tag)
>> +{
>> +	int ret = 0;
>> +	u8 ph;
>> +
>> +	if (src_bits & VFIO_DEVICE_TPH_SRC_NONE)
>> +		*tag = 0;
>> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_DMABUF)
>> +		ret = vfio_pci_dma_buf_get_tph_by_fd(src_hndl, extended,
>> +						     tag, &ph);
> 
> [Severity: Medium]
> Does this path safely access the dmabuf state?
> 
> Looking at the callee:
> vfio_pci_dma_buf_get_tph_by_fd()
>    vfio_pci_dma_buf_get_pci_tph(dmabuf, ...)
> 
> It appears the dmabuf is accessed without acquiring its reservation
> lock (dmabuf->resv). Could this lead to data races when accessing
> the TPH metadata if it changes concurrently?
> 
>> +	else if (src_bits & VFIO_DEVICE_TPH_SRC_CPU_VOLATILE)
> [ ... ]
>> +static int vfio_pci_core_feature_tph_st(struct vfio_pci_core_device *vdev,
>> +				u32 flags,
>> +				struct vfio_device_feature_tph_st __user *arg,
>> +				size_t argsz)
>> +{
>> +	u32 permit_flags = VFIO_DEVICE_TPH_SRC_MASK | VFIO_DEVICE_TPH_EXTENDED;
>> +	struct vfio_device_feature_tph_st tph_st = {0};
>> +	struct pci_dev *pdev = vdev->pdev;
>> +	u32 src_bits, st_size;
>> +	bool stop_on_zero_st;
>> +	u32 *src_hndl = NULL;
>> +	void __user *uptr;
>> +	bool extended;
>> +	int ret, i;
>> +	u16 tag;
>> +
>> +	if (!vdev->tph_permit || vdev->tph_policy == VFIO_PCI_TPH_POLICY_NO_ST)
>> +		return -EOPNOTSUPP;
>> +
>> +	ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_SET,
>> +				 sizeof(tph_st));
>> +	if (ret <= 0)
>> +		return ret;
>> +
>> +	if (copy_from_user(&tph_st, arg, sizeof(tph_st)))
>> +		return -EFAULT;
>> +
>> +	src_bits = tph_st.flags & VFIO_DEVICE_TPH_SRC_MASK;
>> +	if (vdev->tph_policy != VFIO_PCI_TPH_POLICY_LITERAL)
>> +		permit_flags &= ~VFIO_DEVICE_TPH_SRC_LITERAL;
> 
> [Severity: High]
> Is this missing a restriction for the VFIO_PCI_TPH_POLICY_IV_ST
> policy?
> 
> In vfio_pci_core_feature_tph_resolve(), the IV_ST policy strictly
> limits allowed sources to VFIO_DEVICE_TPH_SRC_DMABUF. However, here
> permit_flags is initialized to VFIO_DEVICE_TPH_SRC_MASK and only
> LITERAL is explicitly removed. Could this allow userspace to program
> CPU ST tags when the device policy forbids it?
> 
>> +	if (!(tph_st.flags & permit_flags) || !is_power_of_2(src_bits))
>> +		return -EINVAL;
> 
> [Severity: High]
> Can this bitwise validation be bypassed if a user sets a forbidden
> source flag along with a permitted flag?
> 
> If VFIO_DEVICE_TPH_EXTENDED is included in permit_flags and
> provided in tph_st.flags, the bitwise AND evaluates to non-zero,
> satisfying the first condition. If only one forbidden source bit is
> set, is_power_of_2() also passes, completely bypassing the policy
> restrictions.
> 
> (Note: This same validation pattern appears to be present in
> vfio_pci_core_feature_tph_resolve() as well).
> 
> [Severity: Medium]
> Does this logic ignore undefined or reserved bits in tph_st.flags
> instead of strictly rejecting them?
> 
> If userspace passes uninitialized garbage in the upper bits, the kernel
> won't return -EINVAL, potentially breaking forward compatibility for
> future extensions.
> 
>> +	extended = !!(tph_st.flags & VFIO_DEVICE_TPH_EXTENDED);
> [ ... ]
>> +	for (i = 0; i < tph_st.count; i++) {
>> +		ret = tph_get_st_tag(pdev, src_bits, src_hndl ? src_hndl[i] : 0,
>> +				     extended, &tag);
>> +		if (ret || (stop_on_zero_st && tag == 0))
>> +			break;
>> +		ret = pcie_tph_set_st_entry(pdev, tph_st.start + i, tag);
> 
> [Severity: High]
> Is it safe to program MSI-X TPH entries without ensuring the PCI
> Memory Space Enable (MSE) bit is active?
> 
> While this function ensures the device is in the D0 power state,
> userspace can dynamically disable PCI_COMMAND_MEMORY. Since
> pcie_tph_set_st_entry() performs direct MMIO writes to the BAR,
> could executing this unconditionally on a disabled BAR result in an
> Unsupported Request (UR) and a fatal Machine Check Exception (MCE)?
> 
>> +		if (ret)
>> +			break;
>> +	}
>> +	ret = i;
> 
> [Severity: High]
> Does this unconditional assignment destroy the error code if the
> loop breaks early?
> 
> If tph_get_st_tag() or pcie_tph_set_st_entry() fails on the very
> first iteration, ret will hold a negative error code (like
> -EINVAL). When the loop breaks, ret = i; overwrites it with 0,
> causing the ioctl to silently return success to userspace.
> 
>> +
>> +out:
>> +	up_write(&vdev->memory_lock);
>> +	kfree(src_hndl);
>> +	return ret;
>> +}
>