All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v21 00/13] vfio/pci: Add PCIe TPH support
@ 2026-11-18  9:12 Chengwen Feng
  2026-11-18  9:12 ` [PATCH v21 01/13] PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction Chengwen Feng
                   ` (12 more replies)
  0 siblings, 13 replies; 22+ messages in thread
From: Chengwen Feng @ 2026-11-18  9:12 UTC (permalink / raw)
  To: alex, jgg, helgaas
  Cc: wathsala.vithanage, wei.huang2, zhipingz, wangzhou1, wangyushan12,
	liuyonglong, kvm, linux-pci

This patchset enables userspace-configurable PCIe TPH support for VFIO in
the **device-to-host (D2H)** scenario, bringing performance benefits for
userspace workloads such as DPDK and SPDK.

The patchset is split into two logical parts:
1. The first eight patches fix and refactor core PCI/TPH kernel code to
   export the required helper interfaces.
2. The second five patches implement the corresponding VFIO TPH
   virtualization and uAPI step by step.


Since TLP TPH ST values are not validated by the IOMMU like TLP address
fields, there exists risk of cross-tenant cache interference if untrusted
VM can program arbitrary ST tags. We introduce two-level security
gating:

The first is the VFIO-PCI module parameter tph_policy, defined as:
   Global TPH policy level (0 = default No-ST):
   0 = No-ST mode: ST programming unavailable
   1 = Add IV-ST mode: ST programming supports NONE / CPU sources
   2 = Add DS-ST mode: RESOLVE returns ST for CPU sources;
                       ST programming supports NONE / CPU sources
       Note: In this 'Add DS-ST mode', device-specific ST tables are not
             preserved across PCI resets, userspace must re-establish ST
             state after reset. The kernel cannot validate these ST
             entries, so this mode is not recommended for general
             virtualization usecases.

Note: IV-ST mode is considered safe for VM use because the ST table is
programmed exclusively via the kernel VFIO uAPI. VM cannot directly
write the architected ST table in MSI-X or config space. The hypervisor
(e.g. QEMU) will verify ST, and also when a target CPU transitions from
online to offline, the kernel automatically (add in this patchset) clears
all ST table entries mapped to that CPU, setting those ST values to zero.

The second gate is per-device opt-in. TPH programming is only permitted
after userspace successfully opt-in via the VFIO TPH feature.


Three VFIO device features are introduced in this series:
1. VFIO_DEVICE_FEATURE_TPH: control and query VFIO TPH feature
   capabilities.
2. VFIO_DEVICE_FEATURE_TPH_PROGRAM: batch program architected steering tag
   table entries.
3. VFIO_DEVICE_FEATURE_TPH_RESOLVE: resolve TPH attributes for a given CPU
   source.

The intended userspace programming model is:
1. Read the virtualized PCI TPH capability via VFIO config space access to
   discover underlying hardware TPH support.
2. Invoke VFIO_DEVICE_FEATURE_TPH SET to opt-in and obtain TPH permission.
3. Invoke VFIO_DEVICE_FEATURE_TPH GET to retrieve available TPH
   capabilities; GET returns an error if opt-in has not been performed.
4. Write the TPH control register through VFIO config space to activate
   hardware TPH.
5. Program architected ST table entries via
   VFIO_DEVICE_FEATURE_TPH_PROGRAM if the device exposes architected ST
   table.
6. Query the ST value for a given CPU ID via
   VFIO_DEVICE_FEATURE_TPH_RESOLVE for DS-ST capable devices, userspace
   may then configure the ST through device MMIO or firmware interfaces.


Key implementation details for capability virtualization:
1. The TPH capability register is virtualized according to the active
   tph_policy:
   a. No-ST policy: IV/DS mode bits, ST location and ST size fields are
      masked. Only NO-ST capability is visible to userspace.
   b. IV-ST policy: DS mode bits are masked. Userspace sees NO-ST and
      IV-ST capabilities (if hardware supports IV-ST).
   c. DS-ST policy: capability register is passed through unmodified.
2. Capability bits returned by VFIO_DEVICE_FEATURE_TPH GET follow these
   rules:
   a. If virtualized TPH exposes architected ST table:
      VFIO_DEVICE_TPH_CAP_PROG_ST_NONE and VFIO_DEVICE_TPH_CAP_PROG_ST_CPU
      are reported. These indicate userspace may use TPH_PROGRAM to update
      ST.
   b. If virtualized TPH supports DS mode:
      VFIO_DEVICE_TPH_CAP_RESOLVE_CPU_ST is reported. This indicates
      userspace can resolve CPU-ID associated ST values via TPH_RESOLVE.


Future P2P / DMABUF extension
------------------------------
The uAPI is designed to easy extend to support device-to-device (P2P) and
DMABUF-backed scenarios on top of Zhiping's P2P TPH patchset:
1. Return VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_PH in all tph_policy in
   VFIO_DEVICE_FEATURE_TPH GET, so that userspace could resolve DMABUF's
   TPH PH info
2. Return VFIO_DEVICE_TPH_CAP_PROG_ST_DMABUF if virtualized TPH has the
   architect ST, so that userspace could program ST entries which
   associated to DMABUFs' fd
3. Return VFIO_DEVICE_TPH_CAP_RESOLVE_DMABUF_ST if virtualized TPH
   supports DS mode, so that userspace could retrieve ST value which
   associated to DMABUF's fd
   
---
v21:
- Focus only device-to-host (D2H) scenario
- Refactor the implementation
v20:
- Address almost all comments from Alex about v19, not within Alex's
  scope:
  - Drop LITERAL policy support
  - uAPI changes: cap have been split into fine-grained
  - TPH config access: still based on perm mechanism, but restrict the
    readable and writable length
  - Debugfs add more fields
v19:
- Address all comments from Alex
  - Drop CPU-to-ST translation of PCI/TPH
  - Re-implement vfio part, please refer:
    https://lore.kernel.org/linux-pci/20260626092232.53ed3a7c@shazbot.org/
- Add expose tph_policy via debugfs

Chengwen Feng (13):
  PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction
  PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing
  PCI/TPH: Cache negotiated max requester type at probe time
  PCI/TPH: Refactor pcie_enable_tph and add requester variant
  PCI/TPH: Refactor pcie_tph_get_cpu_st & add requester variant
  PCI/TPH: Add macros to check standard and extended TPH support
  PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming
    helper
  PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST
    _DSM
  vfio/pci: Add basic NO-ST TPH VFIO device feature support
  vfio/pci: Add IV-ST TPH policy and TPH_PROGRAM batch programming
    support
  vfio/pci: Add DS-ST TPH policy and TPH_RESOLVE capability
  vfio/pci: Reset PCIe TPH state on device open and close
  vfio/pci: Expose per-device TPH state via debugfs

 drivers/pci/tph.c                  | 240 +++++++++++------
 drivers/vfio/pci/vfio_pci.c        |  24 ++
 drivers/vfio/pci/vfio_pci_config.c | 175 +++++++++++--
 drivers/vfio/pci/vfio_pci_core.c   | 402 ++++++++++++++++++++++++++++-
 drivers/vfio/pci/vfio_pci_priv.h   |   6 +
 include/linux/pci-tph.h            |  47 +++-
 include/linux/pci.h                |   6 +-
 include/linux/vfio_pci_core.h      |   6 +
 include/uapi/linux/vfio.h          | 129 +++++++++
 9 files changed, 935 insertions(+), 100 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2026-11-18 10:29 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18  9:12 [PATCH v21 00/13] vfio/pci: Add PCIe TPH support Chengwen Feng
2026-11-18  9:12 ` [PATCH v21 01/13] PCI/TPH: Fix pcie_tph_get_st_table_loc() field extraction Chengwen Feng
2026-11-18  9:29   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 02/13] PCI/TPH: Fix tph_enabled concurrent update race by bitfield packing Chengwen Feng
2026-11-18  9:38   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 03/13] PCI/TPH: Cache negotiated max requester type at probe time Chengwen Feng
2026-11-18  9:45   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 04/13] PCI/TPH: Refactor pcie_enable_tph and add requester variant Chengwen Feng
2026-11-18  9:48   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 05/13] PCI/TPH: Refactor pcie_tph_get_cpu_st & " Chengwen Feng
2026-11-18  9:53   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 06/13] PCI/TPH: Add macros to check standard and extended TPH support Chengwen Feng
2026-11-18 10:06   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 07/13] PCI/TPH: Add pcie_tph_set_st_entries() batch ST entry programming helper Chengwen Feng
2026-11-18 10:15   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 08/13] PCI/TPH: Add pci_tph_dsm_supported() helper to detect device TPH ST _DSM Chengwen Feng
2026-11-18 10:29   ` sashiko-bot
2026-11-18  9:12 ` [PATCH v21 09/13] vfio/pci: Add basic NO-ST TPH VFIO device feature support Chengwen Feng
2026-11-18  9:12 ` [PATCH v21 10/13] vfio/pci: Add IV-ST TPH policy and TPH_PROGRAM batch programming support Chengwen Feng
2026-11-18  9:12 ` [PATCH v21 11/13] vfio/pci: Add DS-ST TPH policy and TPH_RESOLVE capability Chengwen Feng
2026-11-18  9:12 ` [PATCH v21 12/13] vfio/pci: Reset PCIe TPH state on device open and close Chengwen Feng
2026-11-18  9:12 ` [PATCH v21 13/13] vfio/pci: Expose per-device TPH state via debugfs Chengwen Feng

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.