On Mon, 24 Aug 2026, Priyank Rathod wrote: > Per PCIe Base Specification r6.0, sec 8.4.4 ("Lane Margining at > Receiver"), PCIe devices operating at 16.0 GT/s (Gen 4) or higher data > rates support the Lane Margining at Receiver Extended Capability > (ID 0x27), and it is mandatory for receivers operating at 64.0 GT/s > (Gen 6) or higher data rates. Lane Margining allows software to > evaluate high-speed link margins by measuring timing and voltage steps > for each individual physical lane and receiver. > > Add driver and debugfs support for PCIe Lane Margining at Receiver: > > - Add Lane Margining at Receiver Extended Capability register > definitions (PCI_EXT_CAP_ID_LMR, PCI_LMR_PORT_CAP, PCI_LMR_PORT_STS, > PCI_LMR_LANE_CTRL, PCI_LMR_LANE_STS) to . > - Add Kconfig option CONFIG_PCIE_LMR (under drivers/pci/pcie/Kconfig) > dependent on DEBUG_FS. > - Implement drivers/pci/pcie/margin.c to probe the capability on Gen4+ > links and expose per-device debugfs entries under: > /sys/kernel/debug/pci/pcie_lmr_/ > providing control over margining enablement, receiver selection, and > execution of timing/voltage margin step commands. Distinguish > between missing mandatory LMR capability on Gen6+ vs optional on > Gen4/Gen5. > - Hook pci_lmr_init() into pci_init_capabilities() during device probe > in drivers/pci/probe.c and pci_lmr_exit() into drivers/pci/remove.c. > - Add kselftest script under tools/testing/selftests/pcie_lmt/pcie_lmt.sh > to test debugfs capability reads, enablement, and stepping. > - Add MAINTAINERS entry for PCIe Lane Margining at Receiver (LMR). > > Signed-off-by: Priyank Rathod > --- > Changes in v6: > - Added kernel documentation under Documentation/PCI/pcie-lmr.rst and indexed in Documentation/PCI/index.rst (Ilpo Järvinen). > - Updated MAINTAINERS with Documentation/PCI/pcie-lmr.rst (Ilpo Järvinen). > - Aligned capability bit naming and comments with PCIe Base Specification r6.0 sec 8.4.4 Table "Report Margining Capabilities Payload" (Ilpo Järvinen). > - Clarified Sample Multiple Receivers concurrency verification and rules across physical lanes in kerneldoc and documentation (Ilpo Järvinen). > - Refactored pci_lmr_run_cmd() to pass struct pci_margin_dev *mdev directly, eliminating redundant NULL checks and using mdev->num_lanes (Ilpo Järvinen). > - Converted PCI config read/write return checking across all helpers to pcibios_err_to_errno() (Ilpo Järvinen). > - Reversed return logic in pci_lmr_demargin_lane() to return early on error (Ilpo Järvinen). > - Refactored margin_lane_step_write() to eliminate bool is_voltage parameter, using command type (LMR_TYPE_TIMING / LMR_TYPE_VOLTAGE) and switch/case with consolidated bounds checks (Ilpo Järvinen). > - Renamed __pci_suspend_lmr_locked() to pci_lmr_disable_locked() to avoid PM terminology confusion and added lockdep_assert_held(&mdev->lock) (Ilpo Järvinen). > - Replaced -EACCES with -EBUSY across debugfs show/write callbacks when margining is inactive (Ilpo Järvinen). > - Clarified comment for active operating link speed check (Gen4+ capability vs dynamically operating speed) in margin_enable_write() (Ilpo Järvinen). > - Added WARN_ON_ONCE(!dev) check in pci_lmr_init() (Ilpo Järvinen). > - Demoted capability detection log message from pci_info to pci_dbg to prevent boot log noise (Ilpo Järvinen). > - Fixed timing step mask extraction in pci_lmr_cache_rx_info() to use 6-bit LMR_TIMING_STEP_MASK (sashiko-bot). > - Resumed runtime PM via pm_runtime_resume_and_get() before performing config space reads in margin_enable_write() (sashiko-bot). > - Switched to pm_runtime_put_sync() during margining teardown (sashiko-bot). > - Link to v5: https://lore.kernel.org/r/20260820-pcie-lmt-v5-1-943b3b0e18bf@google.com > > PCI/pcie: Add PCIe Lane Margining at Receiver (LMR) support > > Per PCIe Base Specification r6.0, section 8.4.4 ("Lane Margining at Receiver"), > PCIe devices operating at 16.0 GT/s (Gen 4) or higher data rates support the > Lane Margining at Receiver Extended Capability (ID 0x27), and it is mandatory > for receivers operating at 64.0 GT/s (Gen 6) or higher data rates. > > Lane Margining allows system software to evaluate high-speed link signal > integrity and margins by measuring timing and voltage steps for each physical > lane and receiver independently. > > This series introduces kernel driver support, debugfs controls, and a > kselftest automation script for PCIe Lane Margining at Receiver (LMR/LMT). > > ============================================================================== > 1. How to Enable & Configure > ============================================================================== > Enable the Kconfig option under PCI support: > CONFIG_PCIE_LMR=y (or =m) > (Depends on CONFIG_PCI and CONFIG_DEBUG_FS) > > Upon boot or device hotplug on Gen4+ links (>= 16.0 GT/s), the driver probes > Extended Capability ID 0x27 and exposes per-device debugfs interfaces: > /sys/kernel/debug/pci/pcie_lmr_::./ > > ============================================================================== > 2. How to Use the Debugfs Interface (Manual Margining) > ============================================================================== > Inspect device-wide margining capabilities and port status: > # Inspect root device LMR capabilities & status > cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/capabilities > cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/port_status > > Enable active Lane Margining on the device: > # Enable Lane Margining state machine > echo 1 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/enable > > Inspect and step individual lanes (e.g. lane0): > # Select target receiver (0 = local receiver, 1..6 = retimers/link partners) > echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/receiver > > # Check available timing and voltage steps for this receiver > cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/caps > cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/num_timing_steps > cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/num_voltage_steps > > # Step timing margin or voltage margin offset > echo 2 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_timing > echo 1 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_voltage > > # Reset margin offset back to nominal (0) > echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_timing > echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_voltage > > Disable Lane Margining when finished: > echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/enable > > ============================================================================== > 3. How to Run Automated Kselftests Using the Test Script > ============================================================================== > An automated kselftest script is included to test capability reads, receiver > selection, and margining commands across all enumerated LMR devices: > > # Run directly as root > sudo ./tools/testing/selftests/pcie_lmt/pcie_lmt.sh > > Or run via the kselftest Makefile harness: > make -C tools/testing/selftests TARGETS=pcie_lmt run_tests > > Sample script output on an LMR-capable device: > pcie_lmt: testing PCIe LMR debugfs entries > pcie_lmt: probing device pcie_lmr_0000:01:00.0 > pcie_lmr_0000:01:00.0: capabilities read OK > pcie_lmr_0000:01:00.0: port_status read OK > pcie_lmr_0000:01:00.0: margining enabled OK > pcie_lmr_0000:01:00.0: testing lane0 > pcie_lmr_0000:01:00.0: testing lane1 > pcie_lmr_0000:01:00.0: margining disabled OK > pcie_lmt [PASS] > > To: Bjorn Helgaas > To: Shuah Khan > Cc: linux-kernel@vger.kernel.org > Cc: linux-pci@vger.kernel.org > Cc: linux-kselftest@vger.kernel.org > Cc: Ilpo Järvinen > > Changes in v5: > - Sorted #include directives alphabetically and added missing includes for bits.h, bitfield.h, cleanup.h, overflow.h, and slab.h (Ilpo Järvinen). > - Converted bitmasks to GENMASK() and BIT() macros and used FIELD_PREP() and FIELD_GET() instead of manual bit shifts (Ilpo Järvinen). > - Added pci_lmr_sts_payload() helper to cleanly extract the status payload byte before applying step and capability masks (Ilpo Järvinen). > - Replaced manual mutex locking sequences with guard(mutex)(&mdev->lock) across show and write callbacks to simplify control flow (Ilpo Järvinen). > - Documented mutex lock protection scope in kerneldoc for struct pci_margin_dev (Ilpo Järvinen). > - Used standard PCI_POSSIBLE_ERROR(), str_yes_no(), and scnprintf() helpers throughout the driver (Ilpo Järvinen). > - Clarified receiver range (0..6 per PCIe r6.0 sec 8.4.4; 7 reserved) in comments and validation checks (Ilpo Järvinen). > - Deduplicated timing and voltage show/write handlers using margin_lane_steps_show() and margin_lane_step_write() (Ilpo Järvinen). > - Placed speed check immediately following pcie_get_speed_cap() and handled PCI_SPEED_UNKNOWN (Ilpo Järvinen). > - Converted lanes in struct pci_margin_dev to a flexible array member with __counted_by(num_lanes) allocated via struct_size() (Ilpo Järvinen). > > Changes in v4: > - Added Sample Multiple Receivers (Bit 5) concurrency verification in margin_lane_timing_write() and margin_lane_voltage_write() per PCIe r6.0 sec 8.4.4, returning -EBUSY if another lane on the same receiver is already margined when simultaneous lane margining is not supported. > - Added active operating link speed verification (PCI_EXP_LNKSTA_CLS >= 16.0 GT/s) in margin_enable_write() before enabling LMR, as LMR commands are physically undefined on links operating at Gen1/Gen2/Gen3 speeds. > - Added fast-path hardware NAK detection in pci_lmr_run_cmd() to return -EOPNOTSUPP immediately if a receiver echoes MTYPE == NO_CMD (0x7) after command issuance rather than waiting 150ms for a timeout. > - Added pci_reset_lmr() hooked into __pci_reset_function_locked() to synchronize software state and demargin on FLR or Secondary Bus Reset. > - Comprehensive NULL pointer checks and array/lane/receiver bounds checks added across all internal helpers and debugfs write handlers. > - Added MAINTAINERS entry for PCIe Lane Margining at Receiver (LMR). > > Changes in v2: > - Fixed NO_CMD (0x7) clearing in pci_lmr_run_cmd() before issuing new commands per PCIe r6.0 sec 8.4.4. > - Protected plane->rx updates with mdev->lock in margin_lane_receiver_write(). > - Corrected Margining Port Capabilities bit definition to PCI_LMR_PORT_CAP_USES_SW_READY (0x0001) in . > - Updated kselftest script (pcie_lmt.sh) to locate LMR debugfs entries. > - Validated integer bounds against LMR_MAX_TIMING_STEP / LMR_MAX_VOLTAGE_STEP before narrowing u8 cast. > - Moved mdev->enabled checks inside mutex_lock(&mdev->lock) to eliminate TOCTOU races. > - Checked return values of all pci_read_config_word() calls, propagating -EIO on failure. > - Eliminated dead store of cap in margin_enable_write(). > - Explicitly checked speed == PCIE_SPEED_64_0GT in pci_lmr_init() to avoid misidentifying PCI_SPEED_UNKNOWN (0xFF) as Gen6. > --- > Documentation/PCI/index.rst | 1 + > Documentation/PCI/pcie-lmr.rst | 171 +++++ > MAINTAINERS | 8 + > drivers/pci/pci-driver.c | 1 + > drivers/pci/pci.c | 4 +- > drivers/pci/pci.h | 12 + > drivers/pci/pcie/Kconfig | 12 + > drivers/pci/pcie/Makefile | 1 + > drivers/pci/pcie/margin.c | 1061 ++++++++++++++++++++++++++ > drivers/pci/probe.c | 1 + > drivers/pci/remove.c | 1 + > include/linux/pci.h | 6 + > include/uapi/linux/pci_regs.h | 18 + > tools/testing/selftests/Makefile | 1 + > tools/testing/selftests/pcie_lmt/Makefile | 3 + > tools/testing/selftests/pcie_lmt/pcie_lmt.sh | 105 +++ > 16 files changed, 1405 insertions(+), 1 deletion(-) > > diff --git a/Documentation/PCI/index.rst b/Documentation/PCI/index.rst > index 5d720d2a415e..9170c98cbf3f 100644 > --- a/Documentation/PCI/index.rst > +++ b/Documentation/PCI/index.rst > @@ -20,3 +20,4 @@ PCI Bus Subsystem > controller/index > boot-interrupts > tph > + pcie-lmr > diff --git a/Documentation/PCI/pcie-lmr.rst b/Documentation/PCI/pcie-lmr.rst > new file mode 100644 > index 000000000000..1ebb8317cc83 > --- /dev/null > +++ b/Documentation/PCI/pcie-lmr.rst > @@ -0,0 +1,171 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +======================================================= > +PCI Express Lane Margining at Receiver (LMR) Subsystem > +======================================================= > + > +:Author: Priyank Rathod > +:Copyright: 2026 Google LLC > + > +Overview > +======== > + > +Lane Margining at Receiver (LMR), specified in the PCI Express Base > +Specification (r6.0+ sec 8.4.4), allows system software to evaluate high-speed > +link physical signal integrity and eye margins. LMR measures available timing > +(jitter/phase) and voltage margin offsets for each physical lane and receiver > +independently while the link is operating in active L0 state. > + > +Lane Margining Extended Capability (ID 0x27) is optional for links operating at > +16.0 GT/s (PCIe Gen 4) and 32.0 GT/s (Gen 5), and is mandatory for receivers > +operating at 64.0 GT/s (Gen 6) and higher. > + > +Target Receivers > +================ > + > +Each physical lane can margin up to 7 distinct receivers per PCIe link: > + > +* **Receiver 0 (Local Receiver)**: The receiver in the immediate link partner. > +* **Receivers 1 to 6 (Retimers)**: Retimer pseudo-ports along the physical link > + (up to 3 retimers, each with upstream and downstream pseudo-ports). > +* **Receiver 7**: Reserved per PCIe Base Specification. > + > +Kernel Configuration > +==================== > + > +Enable the kernel configuration option under PCI support: > + > +.. code-block:: none > + > + CONFIG_PCIE_LMR=y (or =m) > + > +Dependencies: > +* ``CONFIG_PCI`` > +* ``CONFIG_DEBUG_FS`` > + > +Debugfs Interface Guide > +======================= > + > +When an LMR-capable device is enumerated on a Gen4+ link, the kernel exposes > +per-device control and status files under debugfs: > + > +.. code-block:: none > + > + /sys/kernel/debug/pci/pcie_lmr_::./ > + > +Device-Level Attributes > +----------------------- > + > +* ``capabilities`` (read-only): > + Displays the 16-bit Margining Port Capabilities register and whether the > + device uses the Software Ready handshake bit. > + > +* ``port_status`` (read-only): > + Displays the Margining Port Status register, indicating Margining Ready and > + SW Ready states. > + > +* ``enable`` (read-write): > + Enables (``1``) or disables (``0``) Lane Margining on the device. > + Enabling margining locks the link into D0, prevents runtime PM suspend, > + disables ASPM L0s/L1, and verifies that the link is operating at >= 16.0 GT/s. > + Disabling margining restores ASPM and runtime PM, and returns all lanes to > + nominal (demargined) state. > + > +Lane-Level Attributes > +--------------------- > + > +For each physical lane (``lane0``, ``lane1``, ...): > + > +* ``receiver`` (read-write): > + Gets or sets the active target receiver number (``0`` for local receiver, > + ``1..6`` for retimers). Switching receivers automatically demargins previous > + offsets per PCIe single-receiver margining requirements. > + > +* ``caps`` (read-only): > + Reports the target receiver's margining capabilities: > + - Margining uses Driver Software (vs hardware autonomous) > + - Independent Left/Right Timing Margining support > + - Independent Up/Down Voltage Margining support > + - Error Sampler vs Main Sampler > + - Sample Multiple Receivers support > + > +* ``num_timing_steps`` (read-only): > + Maximum timing margin steps supported by the receiver (0..63). > + > +* ``num_voltage_steps`` (read-only): > + Maximum voltage margin steps supported by the receiver (0..127). > + > +* ``margin_timing`` (read-write): > + Applies timing margin step offset (+/-). Writing ``0`` clears timing margin > + back to nominal. > + > +* ``margin_voltage`` (read-write): > + Applies voltage margin step offset (+/-). Writing ``0`` clears voltage margin > + back to nominal. > + > +Manual Margining Example > +======================== > + > +1. Inspect device capabilities and status: > + > +.. code-block:: sh > + > + cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/capabilities > + cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/port_status > + > +2. Enable Lane Margining mode: > + > +.. code-block:: sh > + > + echo 1 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/enable > + > +3. Configure target receiver and inspect step limits on lane 0: > + > +.. code-block:: sh > + > + echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/receiver > + cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/caps > + cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/num_timing_steps > + cat /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/num_voltage_steps > + > +4. Apply timing and voltage margin steps: > + > +.. code-block:: sh > + > + # Step timing margin +2 steps > + echo 2 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_timing > + > + # Step voltage margin +1 step > + echo 1 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_voltage > + > +5. Reset margins back to nominal: > + > +.. code-block:: sh > + > + echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_timing > + echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/lane0/margin_voltage > + > +6. Disable Lane Margining when complete: > + > +.. code-block:: sh > + > + echo 0 > /sys/kernel/debug/pci/pcie_lmr_0000:01:00.0/enable > + > +Automated Testing via Kselftest > +=============================== > + > +The kernel includes an automated kselftest script under > +``tools/testing/selftests/pcie_lmt/pcie_lmt.sh`` to probe, validate, and exercise > +debugfs controls across all enumerated LMR devices. > + > +Run directly as root: > + > +.. code-block:: sh > + > + sudo ./tools/testing/selftests/pcie_lmt/pcie_lmt.sh > + > +Or run via the kselftest test harness: > + > +.. code-block:: sh > + > + make -C tools/testing/selftests TARGETS=pcie_lmt run_tests > diff --git a/MAINTAINERS b/MAINTAINERS > index b7094a616afd..b5deaae11bfe 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -21059,6 +21059,14 @@ F: Documentation/devicetree/bindings/pci/qcom,sa8255p-pcie-ep.yaml > F: drivers/pci/controller/dwc/pcie-qcom-common.c > F: drivers/pci/controller/dwc/pcie-qcom-ep.c > > +PCIE LANE MARGINING AT RECEIVER (LMR) > +M: Priyank Rathod > +L: linux-pci@vger.kernel.org > +S: Maintained > +F: Documentation/PCI/pcie-lmr.rst > +F: drivers/pci/pcie/margin.c > +F: tools/testing/selftests/pcie_lmt/ > + > PCMCIA SUBSYSTEM > M: Dominik Brodowski > S: Odd Fixes > diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c > index f36778e62ac1..17544a7023fc 100644 > --- a/drivers/pci/pci-driver.c > +++ b/drivers/pci/pci-driver.c > @@ -821,6 +821,7 @@ static int pci_pm_suspend(struct device *dev) > * since Coffee Lake, to enter a lower-power PM state. > */ > pci_suspend_ptm(pci_dev); > + pci_suspend_lmr(pci_dev); > > if (pci_has_legacy_pm_support(pci_dev)) > return pci_legacy_suspend(dev, PMSG_SUSPEND); > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 77b17b13ee61..dc9724cb7b4d 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -5145,8 +5145,10 @@ int __pci_reset_function_locked(struct pci_dev *dev) > method = &pci_reset_fn_methods[m]; > pci_dbg(dev, "reset via %s\n", method->name); > rc = method->reset_fn(dev, PCI_RESET_DO_RESET); > - if (!rc) > + if (!rc) { > + pci_reset_lmr(dev); > return 0; > + } > > pci_dbg(dev, "%s failed with %d\n", method->name, rc); > if (rc != -ENOTTY) > diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h > index 4469e1a77f3c..6322a81f9e50 100644 > --- a/drivers/pci/pci.h > +++ b/drivers/pci/pci.h > @@ -1023,6 +1023,18 @@ static inline void pci_no_tph(void) { } > static inline void pci_tph_init(struct pci_dev *dev) { } > #endif > > +#ifdef CONFIG_PCIE_LMR > +void pci_lmr_init(struct pci_dev *dev); > +void pci_lmr_exit(struct pci_dev *dev); > +void pci_suspend_lmr(struct pci_dev *dev); > +void pci_reset_lmr(struct pci_dev *dev); > +#else > +static inline void pci_lmr_init(struct pci_dev *dev) { } > +static inline void pci_lmr_exit(struct pci_dev *dev) { } > +static inline void pci_suspend_lmr(struct pci_dev *dev) { } > +static inline void pci_reset_lmr(struct pci_dev *dev) { } > +#endif > + > #ifdef CONFIG_PCIE_PTM > void pci_ptm_init(struct pci_dev *dev); > void pci_save_ptm_state(struct pci_dev *dev); > diff --git a/drivers/pci/pcie/Kconfig b/drivers/pci/pcie/Kconfig > index 207c2deae35f..3b021ca2fe84 100644 > --- a/drivers/pci/pcie/Kconfig > +++ b/drivers/pci/pcie/Kconfig > @@ -137,6 +137,18 @@ config PCIE_PTM > This is only useful if you have devices that support PTM, but it > is safe to enable even if you don't. > > +config PCIE_LMR > + bool "PCI Express Lane Margining at Receiver Support" > + depends on DEBUG_FS > + help > + This enables the PCI Express Lane Margining at Receiver support. > + Lane Margining allows software to determine the voltage and > + timing margin of each lane on a PCIe link (16.0 GT/s and above). > + The margining data is exposed via debugfs. > + > + This is only useful if you have devices that support lane > + margining, but it is safe to enable even if you don't. > + > config PCIE_EDR > bool "PCI Express Error Disconnect Recover support" > depends on PCIE_DPC && ACPI > diff --git a/drivers/pci/pcie/Makefile b/drivers/pci/pcie/Makefile > index b0b43a18c304..aac45ae0402e 100644 > --- a/drivers/pci/pcie/Makefile > +++ b/drivers/pci/pcie/Makefile > @@ -13,4 +13,5 @@ obj-$(CONFIG_PCIEAER_INJECT) += aer_inject.o > obj-$(CONFIG_PCIE_PME) += pme.o > obj-$(CONFIG_PCIE_DPC) += dpc.o > obj-$(CONFIG_PCIE_PTM) += ptm.o > +obj-$(CONFIG_PCIE_LMR) += margin.o > obj-$(CONFIG_PCIE_EDR) += edr.o > diff --git a/drivers/pci/pcie/margin.c b/drivers/pci/pcie/margin.c > new file mode 100644 > index 000000000000..45646b5952d6 > --- /dev/null > +++ b/drivers/pci/pcie/margin.c > @@ -0,0 +1,1061 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * PCI Express Lane Margining at Receiver > + * > + * Copyright (C) 2026 Google LLC > + * Author: Priyank Rathod > + * > + * Lane Margining at Receiver (PCIe Base Specification r6.0, sec 8.4.4) > + * allows system software to determine the voltage and timing margins of > + * each physical lane on a PCIe link. The Extended Capability (ID 0x27) > + * is available for receivers operating at 16.0 GT/s (Gen4) or higher data > + * rates, and is mandatory for receivers operating at 64.0 GT/s (Gen6) or > + * higher data rates. > + * > + * This driver implements: > + * - Probing Extended Capability ID 0x27 and Margining Port Capabilities. > + * - Managing ASPM L0s/L1 link states during active margining with restoration. > + * - PCIe r6.0 NO_CMD (0x7) clearing handshake per receiver and lane. > + * - Caching receiver capabilities & step counts to avoid DEMARGIN side-effects. > + * - Handling Symmetric vs Independent Left/Right & Up/Down margin steps. > + * - Runtime PM protection (D0 enforcement) during active margining. > + * - Exposing per-device debugfs interfaces under /sys/kernel/debug/pci/. > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "../pci.h" > + > +/* Margin type encodings per PCIe Base Spec r6.0 sec 8.4.4 */ > +#define LMR_TYPE_DEMARGIN 0x0 > +#define LMR_TYPE_REPORT_CAPS 0x1 > +#define LMR_TYPE_REPORT_VOLTAGE_STEPS 0x2 > +#define LMR_TYPE_REPORT_TIMING_STEPS 0x3 > +#define LMR_TYPE_TIMING 0x4 > +#define LMR_TYPE_VOLTAGE 0x5 > +#define LMR_TYPE_NO_CMD 0x7 > + > +/* LMR command timing parameters */ > +#define LMR_CMD_TIMEOUT_MS 150 > +#define LMR_CMD_SLEEP_MIN_US 100 > +#define LMR_CMD_SLEEP_MAX_US 250 > +#define LMR_ENABLE_TIMEOUT_MS 150 > +#define LMR_ENABLE_SLEEP_MIN_US 1000 > +#define LMR_ENABLE_SLEEP_MAX_US 2000 > + > +/* > + * LMR limits: > + * Valid receiver numbers are 0 (local receiver) to 6 (up to 3 retimers) > + * per PCIe Base Specification r6.0 sec 8.4.4. Receiver number 7 is reserved. > + */ > +#define LMR_MAX_LANES 32 > +#define LMR_MAX_RX_NUM 6 > +#define LMR_MAX_TIMING_STEP 63 > +#define LMR_MAX_VOLTAGE_STEP 127 > + > +/* LMR PCIe generation numbers and helper */ > +#define LMR_GEN6 6 > +#define LMR_GEN5 5 > +#define LMR_GEN4 4 > + > +#define LMR_SPEED_TO_GEN(speed) \ > + ((speed) >= PCIE_SPEED_64_0GT ? LMR_GEN6 : \ > + (speed) >= PCIE_SPEED_32_0GT ? LMR_GEN5 : \ > + LMR_GEN4) > + > +/* LMR lane register stride */ > +#define LMR_LANE_REG_STRIDE 4 > + > +/* LMR receivers and directions */ > +#define LMR_RX_LOCAL 0 > +#define LMR_STEP_DIR_INCREASE 1 > +#define LMR_STEP_DIR_DECREASE 0 > + > +/* LMR payload field masks per PCIe Base Spec r6.0 sec 8.4.4 */ > +#define LMR_STEPS_MASK GENMASK(6, 0) > +#define LMR_TIMING_STEP_MASK GENMASK(5, 0) > +#define LMR_TIMING_DIR_MASK BIT(6) > +#define LMR_VOLTAGE_STEP_MASK GENMASK(6, 0) > +#define LMR_VOLTAGE_DIR_MASK BIT(7) > + > +/* > + * Margining Capabilities report bit fields (PCIe Base Spec r6.0 sec 8.4.4, > + * Table "Report Margining Capabilities Payload"): In 8.4.4, my copy of r6.0.1 (and same with r7.0) PCIe spec, I only have one table and that is called: "Table 8-13 Lane Margining" And no search finds "Report Margining Capabilities Payload" table anywhere. ...So I still fail to find it. (AI has tendencity to come up non-existing things, I hope it's not the case here as it would be rather rude to waste reviewers time on chasing non-existing things.) > + * Bit 0: Margining Uses Driver Software (1 = Driver software sequence; 0 = Hardware) > + * Bit 2: Independent Left/Right Timing Margining Supported (1 = Supported; 0 = Symmetric) > + * Bit 3: Independent Up/Down Voltage Margining Supported (1 = Supported; 0 = Symmetric) > + * Bit 4: Margining Error Sampler (1 = Error Sampler; 0 = Main Sampler) > + * Bit 5: Sample Multiple Receivers (1 = Multiple receivers; 0 = Single receiver only) > + */ > +#define LMR_CAP_USES_DRIVER_SW BIT(0) > +#define LMR_CAP_IND_LEFT_RIGHT_TIMING BIT(2) > +#define LMR_CAP_IND_UP_DOWN_VOLTAGE BIT(3) > +#define LMR_CAP_ERROR_SAMPLER BIT(4) > +#define LMR_CAP_SAMPLE_MULTIPLE_RX BIT(5) > + > +/** > + * struct pci_margin_rx_info - Cached Lane Margining receiver capabilities > + * @caps_cached: True if receiver capabilities and step limits are cached > + * @caps: Margining capabilities byte reported by receiver > + * @num_timing_steps: Maximum timing margin steps supported by receiver > + * @num_voltage_steps: Maximum voltage margin steps supported by receiver > + */ > +struct pci_margin_rx_info { > + bool caps_cached; > + u8 caps; > + u8 num_timing_steps; > + u8 num_voltage_steps; > +}; > + > +/** > + * struct pci_margin_lane - Per-lane margining state > + * @mdev: Parent LMR margin device > + * @lane: Physical lane index (0..num_lanes - 1) > + * @rx: Selected target receiver number (0 = local, 1..6 = retimers) > + * @timing_val: Current applied timing margin step offset (+/-) > + * @voltage_val: Current applied voltage margin step offset (+/-) > + * @rx_info: Cached receiver capabilities per receiver number > + */ > +struct pci_margin_lane { > + struct pci_margin_dev *mdev; > + int lane; > + u8 rx; > + int timing_val; > + int voltage_val; > + struct pci_margin_rx_info rx_info[LMR_MAX_RX_NUM + 1]; > +}; > + > +/** > + * struct pci_margin_dev - PCIe Lane Margining device instance > + * @dev: Underlying PCI device > + * @cap: Extended capability offset (PCI_EXT_CAP_ID_LMR) > + * @debugfs: Root debugfs dentry for this device > + * @lock: Mutex protecting LMR hardware access, active margining enablement, > + * target receiver selection, lane margining steps, and ASPM state > + * @enabled: True if Lane Margining is currently enabled > + * @aspm_saved: True if original ASPM configuration has been saved > + * @saved_aspm: Saved ASPM control register bits for the device > + * @saved_parent_aspm: Saved ASPM control register bits for parent bridge > + * @num_lanes: Number of lanes on the link > + * @lanes: Flexible array of per-lane state structures > + */ > +struct pci_margin_dev { > + struct pci_dev *dev; > + u16 cap; > + struct dentry *debugfs; > + struct mutex lock; > + bool enabled; > + bool aspm_saved; > + u16 saved_aspm; > + u16 saved_parent_aspm; > + int num_lanes; > + struct pci_margin_lane lanes[] __counted_by(num_lanes); > +}; > + > +#if IS_ENABLED(CONFIG_DEBUG_FS) > +static DEFINE_MUTEX(pci_debugfs_root_lock); > +static struct dentry *pci_debugfs_root_dir; > + > +static struct dentry *get_pci_debugfs_root(void) > +{ > + mutex_lock(&pci_debugfs_root_lock); > + if (!pci_debugfs_root_dir) > + pci_debugfs_root_dir = debugfs_lookup("pci", NULL); > + if (!pci_debugfs_root_dir) > + pci_debugfs_root_dir = debugfs_create_dir("pci", NULL); > + mutex_unlock(&pci_debugfs_root_lock); > + return pci_debugfs_root_dir; > +} > +#endif > + > +/* > + * pci_lmr_disable_aspm() - Temporarily disable ASPM L0s/L1 during active > + * margining per PCIe Base Spec r6.0 sec 8.4.4, saving original ASPMC bits. > + */ > +static void pci_lmr_disable_aspm(struct pci_margin_dev *mdev) > +{ > + struct pci_dev *dev = mdev->dev; > + struct pci_dev *parent = pci_upstream_bridge(dev); > + u16 ctl; > + > + if (mdev->aspm_saved) > + return; > + > + if (!pcie_capability_read_word(dev, PCI_EXP_LNKCTL, &ctl)) { > + mdev->saved_aspm = ctl & PCI_EXP_LNKCTL_ASPMC; > + pcie_capability_clear_word(dev, PCI_EXP_LNKCTL, PCI_EXP_LNKCTL_ASPMC); > + } > + > + if (parent && pci_is_pcie(parent)) { > + if (!pcie_capability_read_word(parent, PCI_EXP_LNKCTL, &ctl)) { > + mdev->saved_parent_aspm = ctl & PCI_EXP_LNKCTL_ASPMC; > + pcie_capability_clear_word(parent, PCI_EXP_LNKCTL, PCI_EXP_LNKCTL_ASPMC); > + } > + } > + mdev->aspm_saved = true; Since you ignored my previous inquiry, I'm asking again... How exactly you intend to prevent the aspm driver from re-enabling ASPM while this driver wants it to remain off? > +} > + > +/* > + * pci_lmr_restore_aspm() - Restore original ASPM L0s/L1 state when margining > + * is disabled or torn down. > + */ > +static void pci_lmr_restore_aspm(struct pci_margin_dev *mdev) > +{ > + struct pci_dev *dev = mdev->dev; > + struct pci_dev *parent = pci_upstream_bridge(dev); > + > + if (!mdev->aspm_saved) > + return; > + > + pcie_capability_clear_and_set_word(dev, PCI_EXP_LNKCTL, > + PCI_EXP_LNKCTL_ASPMC, > + mdev->saved_aspm); > + if (parent && pci_is_pcie(parent)) > + pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL, > + PCI_EXP_LNKCTL_ASPMC, > + mdev->saved_parent_aspm); > + mdev->aspm_saved = false; > +} > +static ssize_t margin_enable_write(struct file *file, const char __user *user_buf, > + size_t count, loff_t *ppos) > +{ > + struct seq_file *s = file->private_data; > + struct pci_margin_dev *mdev = s->private; > + struct pci_dev *dev = mdev->dev; > + unsigned long timeout; > + u16 sts, cap, lnksta; > + bool enable; > + int ret, i; > + > + ret = kstrtobool_from_user(user_buf, count, &enable); > + if (ret) > + return ret; > + > + guard(mutex)(&mdev->lock); > + > + if (mdev->enabled == enable) > + return count; > + > + if (!enable) { > + pci_lmr_disable_locked(mdev); > + return count; > + } > + > + /* Ensure device is powered (D0) before reading configuration registers */ > + ret = pm_runtime_resume_and_get(&dev->dev); > + if (ret < 0) > + return ret; > + > + /* > + * PCIe r6.0 sec 8.4.4: LMR is physically undefined below 16.0 GT/s. > + * Even if a device supports Gen4+, if the link is currently trained > + * and operating at Gen1..Gen3 speeds (< 16.0 GT/s), reject margining. > + */ > + pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta); > + if ((lnksta & PCI_EXP_LNKSTA_CLS) < PCI_EXP_LNKSTA_CLS_16_0GB) { > + ret = -EOPNOTSUPP; > + goto err_rpm; > + } > + > + ret = pci_read_config_word(dev, mdev->cap + PCI_LMR_PORT_CAP, &cap); > + if (ret != PCIBIOS_SUCCESSFUL) { > + ret = pcibios_err_to_errno(ret); > + goto err_rpm; > + } > + > + /* Disable ASPM L0s/L1 during margining with restoration path */ > + pci_lmr_disable_aspm(mdev); What about the other steps besides ASPM that the spec required to be disabled during Lane Margining? > + /* Ensure link is settled in L0 mode per PCIe r6.0 sec 8.4.4 */ > + usleep_range(2000, 3000); > + > + if (cap & PCI_LMR_PORT_CAP_USES_SW_READY) { > + ret = pci_read_config_word(dev, mdev->cap + PCI_LMR_PORT_STS, &sts); > + if (ret != PCIBIOS_SUCCESSFUL) { > + ret = pcibios_err_to_errno(ret); > + goto err_aspm; > + } > + sts |= PCI_LMR_PORT_STS_SW_READY; > + pci_write_config_word(dev, mdev->cap + PCI_LMR_PORT_STS, sts); > + } > + > + timeout = jiffies + msecs_to_jiffies(LMR_ENABLE_TIMEOUT_MS); > + while (1) { > + ret = pci_read_config_word(dev, mdev->cap + PCI_LMR_PORT_STS, &sts); > + if (ret != PCIBIOS_SUCCESSFUL) { > + ret = pcibios_err_to_errno(ret); > + goto err_sw_ready; > + } > + if (PCI_POSSIBLE_ERROR(sts)) { > + ret = -ENODEV; > + goto err_sw_ready; > + } > + if (sts & PCI_LMR_PORT_STS_MARGIN_READY) > + break; > + if (time_after(jiffies, timeout)) { > + ret = -ETIMEDOUT; > + goto err_sw_ready; > + } > + usleep_range(LMR_ENABLE_SLEEP_MIN_US, LMR_ENABLE_SLEEP_MAX_US); > + } > + > + /* Cache capabilities for configured receiver on all lanes */ > + for (i = 0; i < mdev->num_lanes; i++) { > + ret = pci_lmr_cache_rx_info(&mdev->lanes[i], mdev->lanes[i].rx); > + if (ret) > + goto err_sw_ready; > + } > + mdev->enabled = true; > + return count; > + > +err_sw_ready: > + if (cap & PCI_LMR_PORT_CAP_USES_SW_READY) { > + ret = pci_read_config_word(dev, mdev->cap + PCI_LMR_PORT_STS, &sts); > + if (ret == PCIBIOS_SUCCESSFUL) { > + sts &= ~PCI_LMR_PORT_STS_SW_READY; > + pci_write_config_word(dev, mdev->cap + PCI_LMR_PORT_STS, sts); > + } > + } > +err_aspm: > + pci_lmr_restore_aspm(mdev); > +err_rpm: > + pm_runtime_put_sync(&dev->dev); > + return ret; > +} -- i.