[PATCH v2 0/3] PCI/pcie: Add PCIe Lane Margining at Receiver (LMR) support

[Priyank Rathod <rathodpriyank@google.com>]

Hi Bjorn, Ilpo, and the PCI community,

This patch series introduces upstream Linux kernel support for PCIe Lane Margining
at Receiver (LMR) per PCI Express Base Specification Revision 7.0 / 6.0 (§ 7.7.11
and § 8.4.4).

During the review of the previous v7 standalone LMR submission
(https://lore.kernel.org/linux-pci/20260828-pcie-lmt-v7-1-6012e9e0940a@google.com/),
Ilpo Järvinen pointed out that feature drivers resolving link partners duplicate
link traversal logic that is already present in drivers such as ASPM:

  "This feels like duplicating similar functionality with the aspm driver
   that also wants to infer ends of the link when giving a pci_dev in.
   The aspm driver currently does that within, but it kind of duplicating
   pci_bus. It would be nice to avoid the duplication and have something
   similar for this in PCI core."

In response to this feedback, the implementation is factored into a clean,
modular 3-patch stack:

1. Patch 1/3 (PCI: Add pcie_get_link_endpoints() helper):
   Standardized, race-safe helper in drivers/pci/pci.c and include/linux/pci.h
   to identify both ends of a point-to-point PCIe link. Safely inspects the
   subordinate bus under down_read(&pci_bus_sem) and acquires a reference
   (pci_dev_get()), paired symmetrically with pcie_put_link_endpoints().
   Filters for Function 0 (with ARI support), validates bridge ownership
   (child->self == pdev) to prevent ABA pointer reuse, and handles RCiEP
   and empty downstream ports (-ENODEV).

2. Patch 2/3 (PCI/ASPM: Add pci_aspm_inhibit() helper for temporary link state suppression):
   Reference-counted mechanism (aspm_inhibit_cnt) in drivers/pci/pcie/aspm.c
   to temporarily disable ASPM state transitions (L0s, L1, L1SS) on an active link.
   Guarantees callers hold down_read(&pci_bus_sem) (via lockdep_assert_held_read),
   eliminating deadlocks and preventing power-saving transitions while hardware
   link characterization is in progress.

3. Patch 3/3 (PCI/pcie: Add PCIe Lane Margining at Receiver (LMR) support):
   Implements the core LMR driver (CONFIG_PCIE_LMR) in drivers/pci/pcie/margin.c
   exposing a debugfs interface under /sys/kernel/debug/pci/pcie_lmr_<dev>/.
   Strictly self-contained: 0 modified lines in drivers/pci/pci.c, pci-driver.c,
   or core drivers/pci/pci.h. Exposes clean lifecycle hooks in drivers/pci/probe.c
   and drivers/pci/remove.c, supports 2D timing and voltage margining across
   physical receivers (0..6), enforces Function 0 endpoint filtering (§ 7.7.11),
   and includes an exhaustive kselftest suite (tools/testing/selftests/pcie_lmt/).

- Capability Discovery (§ 7.7.11): Probed via Extended Capability ID 0x27 on
  Gen4+ (>= 16 GT/s) physical links. Differentiates optional support on Gen4/Gen5
  from mandatory capability presence on Gen6+ (>= 64 GT/s).
- Multi-Function Device Scope (§ 7.7.11): Enforces that capability registration
  is strictly restricted to Function 0 on Endpoints / Upstream Ports.
- Payload Decoding (Table 4-77, Cmd 88h): Accurately decodes Margin Control
  Capabilities (Bits 0..4); preserves Bits 7:5 as reserved 0b.
- Direction Encodings (§ 4.2.18.1.2): Enforces that Left/Right and Up/Down
  direction bits remain 0b for symmetric receivers (Bits 6 & 7 reserved 0b).
- Multi-Receiver Concurrency (§ 4.2.18.2): Governed strictly by MIndErrorSampler.
  If MIndErrorSampler == 0b (main data sampler), at most one receiver across the
  link is margined at a time.
- Autonomous Speed & Width Transitions (§ 7.5.3.7, § 7.5.3.17, § 8.4.4): Software
  sequencing disables Downstream Component before Upstream Component on enablement,
  and restores Upstream Component before Downstream Component on teardown.
  Restoration is unconditional via pcie_capability_clear_and_set_word()
  (serialized by pci_lock), eliminating trylock failure traps.

Ingress paths (margin_enable_write and pci_lmr_exit) strictly follow the canonical
PCI locking DAG:
  down_read(&pci_bus_sem)
    -> pci_dev_lock(downstream_port)
      -> pci_dev_lock(upstream_port)
        -> mutex_lock(&mdev->lock)
          -> spin_lock_irqsave(&aspm_lock)

Key Concurrency Guarantees:
1. Hot-Swap Identity Invariance: When disabling LMR, dynamic endpoint resolution
   is completely bypassed. Teardown binds strictly to the session-saved mdev->partner,
   which is read and pinned with pci_dev_get() under mutex_lock(&mdev->lock).
2. Surprise Removal & Fault Hardening: Hardware register accesses check
   pci_dev_is_disconnected() and PCI_POSSIBLE_ERROR() guards, preventing MMIO
   or config space bus aborts when hardware is pulled.
3. Power Management Synchronization: Both link partners are pinned in D0 using
   pm_runtime_resume_and_get() during enablement, and symmetrically released via
   pm_runtime_put() on teardown. System suspend hooks into pci_pm_prepare() where
   all devices in the hierarchy are guaranteed to be in D0.
4. Security Teardown Order: In pci_destroy_dev(), pci_lmr_exit() is called after
   pci_tsm_destroy(), preserving D0 state for TSM link operations (PCIe IDE unbind
   and SPDM cryptographic session teardown).

- Build: Clean compile on x86_64 and ARM64; make W=1 drivers/pci/ (0 warnings).
- Linters: checkpatch.pl clean across all 3 patches (0 errors, 0 warnings).
- Kselftest: tools/testing/selftests/pcie_lmt/pcie_lmt.sh expanded to 287 lines,
  covering positive and negative boundary tests (syntax clean with bash -n).
- Pre-Commit AI Review (Sashiko AI):
  * Patch 1 (f34fdf7f5c6b): Reviewed — Clean (0 issues).
  * Patch 2 (6adc680a727f): Reviewed — Clean (0 issues).
  * Patch 3 (c62b5f92af63): Reviewed — Clean (0 issues, "No issues found.").

Signed-off-by: Priyank Rathod <rathodpriyank@google.com>
---
Changes in v2:
- Decomposed monolithic LMR driver into a 3-patch stack with dedicated core helpers:
  * Patch 1: PCI core link endpoint discovery helper (pcie_get_link_endpoints()).
  * Patch 2: Core ASPM temporary inhibition helper (pci_aspm_inhibit()).
  * Patch 3: PCIe Lane Margining at Receiver driver and debugfs interface.
- Core Scope Discipline: Confined LMR logic exclusively to drivers/pci/pcie/margin.c
  and include/linux/pci.h forward declarations (0 modified lines in pci.c/pci-driver.c).
- Endpoint Discovery Hardening:
  * Handled RCiEP and empty downstream buses returning -ENODEV.
  * Added Function 0 resolution (pcie_find_link_upstream_func0()) supporting PCIe ARI.
  * Added bridge ownership verification (child->self == pdev) to prevent ABA races.
  * Added symmetric pcie_put_link_endpoints() to balance pci_dev_get() references.
- ASPM Inhibit Hardening:
  * Added lockdep_assert_held_read(&pci_bus_sem) to pci_aspm_inhibit_locked().
  * Added reference counting (aspm_inhibit_cnt) to support nested/concurrent calls.
- Hot-Swap & Concurrency Protections:
  * Eliminated hot-swap identity mismatch in margin_enable_write() by binding teardown
    strictly to session-saved mdev->partner instead of re-evaluating endpoints.
  * Added mutex protection when reading and pinning mdev->partner to eliminate UAF races.
  * Converted autonomous speed/width restoration to unconditional register write via
    pcie_capability_clear_and_set_word(), eliminating trylock failure degradation.
- Specification Alignments (PCIe Base Spec r7.0):
  * Aligned Extended Capability ID 0x27 to § 7.7.11 and § 8.4.4.
  * Enforced Function 0 restriction on Endpoints / Upstream Ports (§ 7.7.11).
  * Preserved Reserved Bits 7:5 in Command 88h capabilities response.
  * Implemented MIndErrorSampler-based concurrency for multi-receiver links (§ 4.2.18.2).
  * Corrected direction bit encoding to 0b for symmetric receivers (§ 4.2.18.1.2).
- Kselftest Overhaul:
  * Extended test coverage from 105 to 287 lines, adding comprehensive boundary
    and negative validation tests.
- Link to v1: https://lore.kernel.org/r/20260831-pcie-link-endpoints-v1-1-32c2fd893e9e@google.com
- Link to v7 (monolithic LMR): https://lore.kernel.org/linux-pci/20260828-pcie-lmt-v7-1-6012e9e0940a@google.com/

---
Priyank Rathod (3):
      PCI: Add pcie_get_link_endpoints() helper
      PCI/ASPM: Add pci_aspm_inhibit() helper for temporary link state suppression
      PCI/pcie: Add PCIe Lane Margining at Receiver (LMR) support

 Documentation/PCI/index.rst                  |    1 +
 Documentation/PCI/pcie-lmr.rst               |  174 +++
 MAINTAINERS                                  |    8 +
 drivers/pci/pci.c                            |  152 +++
 drivers/pci/pci.h                            |    8 +
 drivers/pci/pcie/Kconfig                     |   12 +
 drivers/pci/pcie/Makefile                    |    1 +
 drivers/pci/pcie/aspm.c                      |  100 ++
 drivers/pci/pcie/margin.c                    | 1592 ++++++++++++++++++++++++++
 drivers/pci/probe.c                          |    1 +
 drivers/pci/remove.c                         |    2 +-
 include/linux/pci.h                          |   17 +
 include/uapi/linux/pci_regs.h                |   18 +
 tools/testing/selftests/Makefile             |    1 +
 tools/testing/selftests/pcie_lmt/Makefile    |    3 +
 tools/testing/selftests/pcie_lmt/pcie_lmt.sh |  287 +++++
 16 files changed, 2376 insertions(+), 1 deletion(-)
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260831-pcie-link-endpoints-978e100d5d06

Best regards,
-- 
Priyank Rathod <rathodpriyank@google.com>