Linux PCI subsystem development
 help / color / mirror / Atom feed
* [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space
@ 2026-11-16  9:35 Matt Thompson via B4 Relay
  2026-11-16  9:35 ` [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down Matt Thompson via B4 Relay
                   ` (5 more replies)
  0 siblings, 6 replies; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

On a RockPro64 v2.1 with a Kintex-7 FPGA endpoint, a downstream vendor
ID read during a PCI rescan raised a synchronous external abort
(ESR 0x96000210). The rescan task exited with IRQs disabled and the
watchdog reported a lockup.

Patches 1 through 3 retain the downstream link check, the guarded
arm64 read fixup, and the AXI link-down latch handling. The fixup
handles current-EL synchronous external read aborts at the annotated
instructions.

Testing repeated re-enable, retrain, and rescan exposed three further
failures:

- PCI runtime resume replayed a saved Link Disable bit after userspace
  cleared it. Patch 4 resumes the port before a sysfs write to Link
  Control while holding the configuration access runtime PM reference.
- The PHY could reach L0 while the data link stayed in initialization.
  Patch 5 retains an early retrain request and issues it once if a
  later downstream access finds initialization still pending.
- The Rockchip PCIe client correctable-error interrupt fired at about
  one million interrupts per second on CPU 0 and starved RCU. Patch 6
  masks that controller-local notification. The standard AER service
  remains on the root port's separate interrupt.

Validation used upstream commit 97d73c825122, these six patches, and
Igor Velkov's separately posted probe and resume retry patch [1]. The
combined source matches the audit branch [2]. The clean kernel
completed 600 disable, re-enable, retrain, and rescan cycles with
runtime PM set to auto. Each cycle found the FPGA without a driver
rebind or a second userspace retrain request. The board stayed on the
same boot, the client IRQ count remained zero, and the kernel logged
no stall or SError. This test did not reproduce the asynchronous SError
that Igor reported on Helios64 [3].

[1] https://lore.kernel.org/r/20261116-rockchip-pcie-resume-v2-1-a633e279be97@iav.lv
[2] https://github.com/boondocklabs/linux/commit/bb7c4f9ee7ed33fc32832060fdecb9115c8e9630
[3] https://lore.kernel.org/r/20261116-rockchip-host-rfc-v2-test-1@iav.lv

---
Changes in v3:
- Add the Fixes tag to the downstream link-state check.
- Resume a suspended downstream port before sysfs Link Control writes.
- Defer early retrain requests when the PHY is still training.
- Mask the Rockchip client correctable-error interrupt.
- Record the 600-cycle RockPro64 test with Igor's separate patch applied.
- Link to v2: https://lore.kernel.org/r/20261113-rfc-b4-v2-0-d6b14f95c1d6@boondocker.io

Changes in v2:
- Restrict the MMIO read fixup to the SEA path and exclude its
  exception-table type from user-access classification.
- Look up the guarded instruction once on the SEA path.
- Use the standard arm64 register-number encoding helpers in
  the exception-table macro (Sashiko review).
- Link to v1: https://lore.kernel.org/r/20261113-rfc-b4-v1-0-b91dcb502250@boondocker.io

---
Matt Thompson (6):
      PCI: rockchip-host: Skip downstream config access while link is down
      arm64: Allow synchronous abort fixups for guarded MMIO reads
      PCI: rockchip-host: Recover downstream configuration access
      PCI: Resume downstream ports before sysfs Link Control writes
      PCI: rockchip-host: Defer retraining during initial link training
      PCI: rockchip-host: Mask the client correctable error interrupt

 arch/arm64/include/asm/asm-extable.h        |  15 ++++
 arch/arm64/mm/extable.c                     |  20 +++++
 arch/arm64/mm/fault.c                       |   4 +
 drivers/pci/controller/pcie-rockchip-host.c | 119 +++++++++++++++++++++++++++-
 drivers/pci/controller/pcie-rockchip.h      |   4 +
 drivers/pci/pci-sysfs.c                     |  16 ++++
 6 files changed, 176 insertions(+), 2 deletions(-)
---
base-commit: 97d73c8251226cf27cc6656b4c542869bb8c69fa
change-id: 20261113-rfc-b4-6b795b536d9a

Best regards,
--  
Matt Thompson <matt@boondocker.io>



^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down
  2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
@ 2026-11-16  9:35 ` Matt Thompson via B4 Relay
  2026-11-16  9:46   ` sashiko-bot
  2026-11-16  9:35 ` [PATCH RFC v3 2/6] arm64: Allow synchronous abort fixups for guarded MMIO reads Matt Thompson via B4 Relay
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

From: Matt Thompson <matt@boondocker.io>

On a RockPro64, a PCI bus rescan with the downstream FPGA unavailable
faulted in rockchip_pcie_rd_conf() during a vendor ID read. The rescan
task exited with interrupts disabled and the watchdog reported a lockup.

Check the controller's link-up and link-training status before issuing
downstream config requests. Return PCIBIOS_DEVICE_NOT_FOUND while the
link is unavailable. Root port config access still uses the local
controller registers.

This check cannot prevent the link from dropping during the transaction.

Fixes: e77f847df54c ("PCI: rockchip: Add Rockchip PCIe controller support")
Signed-off-by: Matt Thompson <matt@boondocker.io>
---
 drivers/pci/controller/pcie-rockchip-host.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
index 5b071bd..35105c1 100644
--- a/drivers/pci/controller/pcie-rockchip-host.c
+++ b/drivers/pci/controller/pcie-rockchip-host.c
@@ -143,12 +143,23 @@ static int rockchip_pcie_wr_own_conf(struct rockchip_pcie *rockchip,
 	return PCIBIOS_SUCCESSFUL;
 }
 
+static bool rockchip_pcie_downstream_ready(struct rockchip_pcie *rockchip)
+{
+	return PCIE_LINK_UP(rockchip_pcie_read(rockchip, PCIE_CLIENT_BASIC_STATUS1)) &&
+	       PCIE_LINK_TRAINING_DONE(rockchip_pcie_read(rockchip, PCIE_CORE_CTRL));
+}
+
 static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
 				       struct pci_bus *bus, u32 devfn,
 				       int where, int size, u32 *val)
 {
 	void __iomem *addr;
 
+	if (!rockchip_pcie_downstream_ready(rockchip)) {
+		*val = ~0U;
+		return PCIBIOS_DEVICE_NOT_FOUND;
+	}
+
 	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
 
 	if (!IS_ALIGNED((uintptr_t)addr, size)) {
@@ -182,6 +193,9 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
 {
 	void __iomem *addr;
 
+	if (!rockchip_pcie_downstream_ready(rockchip))
+		return PCIBIOS_DEVICE_NOT_FOUND;
+
 	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
 
 	if (!IS_ALIGNED((uintptr_t)addr, size))

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH RFC v3 2/6] arm64: Allow synchronous abort fixups for guarded MMIO reads
  2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
  2026-11-16  9:35 ` [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down Matt Thompson via B4 Relay
@ 2026-11-16  9:35 ` Matt Thompson via B4 Relay
  2026-11-16  9:47   ` sashiko-bot
  2026-11-16  9:35 ` [PATCH RFC v3 3/6] PCI: rockchip-host: Recover downstream configuration access Matt Thompson via B4 Relay
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

From: Matt Thompson <matt@boondocker.io>

An RK3399 PCI configuration read can raise a synchronous external data
abort if it does not get a usable response. Allow an explicitly annotated
MMIO load to return an error in this case.

Match only a current-EL synchronous external abort on a read instruction
with the new exception table type. Keep existing handling for other
faults, including asynchronous SError and translation table walk aborts.

Signed-off-by: Matt Thompson <matt@boondocker.io>
---
 arch/arm64/include/asm/asm-extable.h | 15 +++++++++++++++
 arch/arm64/mm/extable.c              | 20 ++++++++++++++++++++
 arch/arm64/mm/fault.c                |  4 ++++
 3 files changed, 39 insertions(+)

diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h
index d67e2fd..ac78167 100644
--- a/arch/arm64/include/asm/asm-extable.h
+++ b/arch/arm64/include/asm/asm-extable.h
@@ -11,6 +11,7 @@
 #define EX_TYPE_KACCESS_ERR_ZERO	3
 #define EX_TYPE_UACCESS_CPY		4
 #define EX_TYPE_LOAD_UNALIGNED_ZEROPAD	5
+#define EX_TYPE_MMIO_READ_ERR_ZERO	6
 
 /* Data fields for EX_TYPE_UACCESS_ERR_ZERO */
 #define EX_DATA_REG_ERR_SHIFT	0
@@ -80,6 +81,11 @@
 #else /* __ASSEMBLER__ */
 
 #include <linux/stringify.h>
+#include <linux/types.h>
+
+struct pt_regs;
+
+bool fixup_sea_exception(struct pt_regs *regs, unsigned long esr);
 
 #define __ASM_EXTABLE_RAW(insn, fixup, type, data)	\
 	".pushsection	__ex_table, \"a\"\n"		\
@@ -111,6 +117,15 @@
 			    EX_DATA_REG(ZERO, zero)			\
 			  ")")
 
+#define _ASM_EXTABLE_MMIO_READ_ERR_ZERO(insn, fixup, err, zero)		\
+	__DEFINE_ASM_GPR_NUMS						\
+	__ASM_EXTABLE_RAW(#insn, #fixup,				\
+			  __stringify(EX_TYPE_MMIO_READ_ERR_ZERO),	\
+			  "("						\
+			    EX_DATA_REG(ERR, err) " | "			\
+			    EX_DATA_REG(ZERO, zero)			\
+			  ")")
+
 #define _ASM_EXTABLE_UACCESS_ERR(insn, fixup, err)			\
 	_ASM_EXTABLE_UACCESS_ERR_ZERO(insn, fixup, err, wzr)
 
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index 299a244..dcb84d8 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -28,6 +28,8 @@ bool insn_may_access_user(unsigned long addr, unsigned long esr)
 		return false;
 
 	switch (ex->type) {
+	case EX_TYPE_MMIO_READ_ERR_ZERO:
+		return false;
 	case EX_TYPE_UACCESS_CPY:
 		return cpy_faulted_on_uaccess(ex, esr);
 	default:
@@ -101,6 +103,8 @@ bool fixup_exception(struct pt_regs *regs, unsigned long esr)
 	case EX_TYPE_UACCESS_ERR_ZERO:
 	case EX_TYPE_KACCESS_ERR_ZERO:
 		return ex_handler_uaccess_err_zero(ex, regs);
+	case EX_TYPE_MMIO_READ_ERR_ZERO:
+		return false;
 	case EX_TYPE_UACCESS_CPY:
 		return ex_handler_uaccess_cpy(ex, regs, esr);
 	case EX_TYPE_LOAD_UNALIGNED_ZEROPAD:
@@ -109,3 +113,19 @@ bool fixup_exception(struct pt_regs *regs, unsigned long esr)
 
 	BUG();
 }
+
+bool fixup_sea_exception(struct pt_regs *regs, unsigned long esr)
+{
+	const struct exception_table_entry *ex;
+
+	if (ESR_ELx_EC(esr) != ESR_ELx_EC_DABT_CUR ||
+	    (esr & ESR_ELx_FSC) != ESR_ELx_FSC_EXTABT ||
+	    (esr & ESR_ELx_WNR))
+		return false;
+
+	ex = search_exception_tables(instruction_pointer(regs));
+	if (!ex || ex->type != EX_TYPE_MMIO_READ_ERR_ZERO)
+		return false;
+
+	return ex_handler_uaccess_err_zero(ex, regs);
+}
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index b227368..ebe8054 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -30,6 +30,7 @@
 #include <linux/hugetlb.h>
 
 #include <asm/acpi.h>
+#include <asm/asm-extable.h>
 #include <asm/bug.h>
 #include <asm/cmpxchg.h>
 #include <asm/cpufeature.h>
@@ -879,6 +880,9 @@ static int do_sea(unsigned long far, unsigned long esr, struct pt_regs *regs)
 	const struct fault_info *inf;
 	unsigned long siaddr;
 
+	if (fixup_sea_exception(regs, esr))
+		return 0;
+
 	inf = esr_to_fault_info(esr);
 
 	if (user_mode(regs) && apei_claim_sea(regs) == 0) {

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH RFC v3 3/6] PCI: rockchip-host: Recover downstream configuration access
  2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
  2026-11-16  9:35 ` [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down Matt Thompson via B4 Relay
  2026-11-16  9:35 ` [PATCH RFC v3 2/6] arm64: Allow synchronous abort fixups for guarded MMIO reads Matt Thompson via B4 Relay
@ 2026-11-16  9:35 ` Matt Thompson via B4 Relay
  2026-11-16  9:52   ` sashiko-bot
  2026-11-16  9:35 ` [PATCH RFC v3 4/6] PCI: Resume downstream ports before sysfs Link Control writes Matt Thompson via B4 Relay
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

From: Matt Thompson <matt@boondocker.io>

The link status can change between the readiness check and a downstream
PCI configuration read. A link that reports up can also fail to return
a usable completion.

On arm64, annotate downstream loads for all three access widths so a
synchronous external abort returns PCIBIOS_DEVICE_NOT_FOUND with all-ones
data. Keep little-endian conversion and I/O read ordering on success.

After retraining, the controller can continue to block configuration
traffic until its AXI link-down latch is cleared. Clear the latch before
downstream config reads and writes once the link readiness checks pass.

Link: https://lore.kernel.org/linux-pci/20261024-rockchip-pcie-resume-v1-2-58e54a5cf5f1@iav.lv/

Signed-off-by: Matt Thompson <matt@boondocker.io>
---
 drivers/pci/controller/pcie-rockchip-host.c | 62 +++++++++++++++++++++++++++--
 drivers/pci/controller/pcie-rockchip.h      |  1 +
 2 files changed, 59 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
index 35105c1..64ac6f4 100644
--- a/drivers/pci/controller/pcie-rockchip-host.c
+++ b/drivers/pci/controller/pcie-rockchip-host.c
@@ -29,6 +29,10 @@
 #include "../pci.h"
 #include "pcie-rockchip.h"
 
+#ifdef CONFIG_ARM64
+#include <asm/asm-extable.h>
+#endif
+
 static void rockchip_pcie_enable_bw_int(struct rockchip_pcie *rockchip)
 {
 	u32 status;
@@ -149,16 +153,36 @@ static bool rockchip_pcie_downstream_ready(struct rockchip_pcie *rockchip)
 	       PCIE_LINK_TRAINING_DONE(rockchip_pcie_read(rockchip, PCIE_CORE_CTRL));
 }
 
+#ifdef CONFIG_ARM64
+/*
+ * An absent completion can raise a synchronous external abort even when
+ * the link status indicated up immediately before the config transaction.
+ */
+#define rockchip_pcie_cfg_load(insn, addr, val)			\
+	({								\
+		int err = 0;					\
+		asm volatile("1: " #insn " %w0, [%2]\n"			\
+			     "2:\n"					\
+			     _ASM_EXTABLE_MMIO_READ_ERR_ZERO(1b, 2b,	\
+							    %w1, %w0)	\
+			     : "=&r" (val), "+r" (err)			\
+			     : "r" (addr) : "memory");			\
+		err;							\
+	})
+#endif
+
 static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
 				       struct pci_bus *bus, u32 devfn,
 				       int where, int size, u32 *val)
 {
 	void __iomem *addr;
+#ifdef CONFIG_ARM64
+	u32 data;
+	int err;
+#endif
 
-	if (!rockchip_pcie_downstream_ready(rockchip)) {
-		*val = ~0U;
-		return PCIBIOS_DEVICE_NOT_FOUND;
-	}
+	if (!rockchip_pcie_downstream_ready(rockchip))
+		goto absent;
 
 	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
 
@@ -174,6 +198,28 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
 		rockchip_pcie_cfg_configuration_accesses(rockchip,
 						AXI_WRAPPER_TYPE1_CFG);
 
+	/* Clear the AXI link-down latch before downstream config traffic. */
+	rockchip_pcie_write(rockchip, 0, PCIE_CORE_AXI_LINK_DOWN);
+
+#ifdef CONFIG_ARM64
+	if (size == 4) {
+		err = rockchip_pcie_cfg_load(ldr, addr, data);
+		*val = le32_to_cpu((__force __le32)data);
+	} else if (size == 2) {
+		err = rockchip_pcie_cfg_load(ldrh, addr, data);
+		*val = le16_to_cpu((__force __le16)data);
+	} else if (size == 1) {
+		err = rockchip_pcie_cfg_load(ldrb, addr, data);
+		*val = data;
+	} else {
+		*val = 0;
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+	}
+	if (err)
+		goto absent;
+	/* Preserve the I/O read ordering of readl/readw/readb. */
+	__io_ar(*val);
+#else
 	if (size == 4) {
 		*val = readl(addr);
 	} else if (size == 2) {
@@ -184,7 +230,12 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
 		*val = 0;
 		return PCIBIOS_BAD_REGISTER_NUMBER;
 	}
+#endif
 	return PCIBIOS_SUCCESSFUL;
+
+absent:
+	*val = ~0U;
+	return PCIBIOS_DEVICE_NOT_FOUND;
 }
 
 static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
@@ -208,6 +259,9 @@ static int rockchip_pcie_wr_other_conf(struct rockchip_pcie *rockchip,
 		rockchip_pcie_cfg_configuration_accesses(rockchip,
 						AXI_WRAPPER_TYPE1_CFG);
 
+	/* Clear the AXI link-down latch before downstream config traffic. */
+	rockchip_pcie_write(rockchip, 0, PCIE_CORE_AXI_LINK_DOWN);
+
 	if (size == 4)
 		writel(val, addr);
 	else if (size == 2)
diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
index b5da156..1cc3621 100644
--- a/drivers/pci/controller/pcie-rockchip.h
+++ b/drivers/pci/controller/pcie-rockchip.h
@@ -164,6 +164,7 @@
 #define MIN_AXI_ADDR_BITS_PASSED		8
 #define PCIE_ADDR_MASK			GENMASK_ULL(63, MIN_AXI_ADDR_BITS_PASSED)
 #define PCIE_CORE_AXI_CONF_BASE		0xc00000
+#define PCIE_CORE_AXI_LINK_DOWN		(PCIE_CORE_AXI_CONF_BASE + 0x824)
 #define PCIE_CORE_OB_REGION_ADDR0	(PCIE_CORE_AXI_CONF_BASE + 0x0)
 #define   PCIE_CORE_OB_REGION_ADDR0_NUM_BITS	0x3f
 #define   PCIE_CORE_OB_REGION_ADDR0_LO_ADDR	PCIE_ADDR_MASK

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH RFC v3 4/6] PCI: Resume downstream ports before sysfs Link Control writes
  2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
                   ` (2 preceding siblings ...)
  2026-11-16  9:35 ` [PATCH RFC v3 3/6] PCI: rockchip-host: Recover downstream configuration access Matt Thompson via B4 Relay
@ 2026-11-16  9:35 ` Matt Thompson via B4 Relay
  2026-11-16 10:04   ` sashiko-bot
  2026-11-16  9:35 ` [PATCH RFC v3 5/6] PCI: rockchip-host: Defer retraining during initial link training Matt Thompson via B4 Relay
  2026-11-16  9:35 ` [PATCH RFC v3 6/6] PCI: rockchip-host: Mask the client correctable error interrupt Matt Thompson via B4 Relay
  5 siblings, 1 reply; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

From: Matt Thompson <matt@boondocker.io>

Configuration space can remain accessible while a port is runtime
suspended. A sysfs Link Control write can then reach hardware before
runtime resume restores the saved state. On RockPro64 a rescan resumed
the port and restored the saved Link Disable bit after userspace had
cleared it, leaving the endpoint absent.

Resume a runtime-suspended downstream port before writing overlapping
Link Control bytes. The configuration access already holds the runtime
PM reference across the write.

Signed-off-by: Matt Thompson <matt@boondocker.io>
---
 drivers/pci/pci-sysfs.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 6669e9a..ff47ea1 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -786,6 +786,7 @@ static ssize_t pci_write_config(struct file *filp, struct kobject *kobj,
 	struct pci_dev *dev = to_pci_dev(kobj_to_dev(kobj));
 	unsigned int size = count;
 	loff_t init_off = off;
+	loff_t lnkctl = pci_pcie_cap(dev) + PCI_EXP_LNKCTL;
 	u8 *data = (u8 *) buf;
 	int ret;
 
@@ -809,6 +810,21 @@ static ssize_t pci_write_config(struct file *filp, struct kobject *kobj,
 
 	pci_config_pm_runtime_get(dev);
 
+	/*
+	 * Link Control can be written while config space remains accessible in
+	 * runtime suspend. Restore saved state first so a later resume cannot
+	 * replace the new link setting with the saved value.
+	 */
+	if (count && pci_is_pcie(dev) && pcie_downstream_port(dev) &&
+	    pm_runtime_suspended(&dev->dev) &&
+	    off < lnkctl + sizeof(u16) && off + count > lnkctl) {
+		ret = pm_runtime_resume(&dev->dev);
+		if (ret < 0) {
+			pci_config_pm_runtime_put(dev);
+			return ret;
+		}
+	}
+
 	if ((off & 1) && size) {
 		pci_user_write_config_byte(dev, off, data[off - init_off]);
 		off++;

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH RFC v3 5/6] PCI: rockchip-host: Defer retraining during initial link training
  2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
                   ` (3 preceding siblings ...)
  2026-11-16  9:35 ` [PATCH RFC v3 4/6] PCI: Resume downstream ports before sysfs Link Control writes Matt Thompson via B4 Relay
@ 2026-11-16  9:35 ` Matt Thompson via B4 Relay
  2026-11-16  9:51   ` sashiko-bot
  2026-11-16  9:35 ` [PATCH RFC v3 6/6] PCI: rockchip-host: Mask the client correctable error interrupt Matt Thompson via B4 Relay
  5 siblings, 1 reply; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

From: Matt Thompson <matt@boondocker.io>

Clearing Link Disable starts initial training. A retrain request can
arrive while the PHY is still in Detect, when it cannot recover a data
link that later stalls in initialization. On RockPro64 the link could
reach L0 while the data link remained unable to pass configuration
traffic.

Remember an early request and give initialization 50 ms to finish.
If a later downstream access finds the PHY ready with data link
initialization still pending, issue the saved request once and leave
configuration access blocked until the data link becomes ready.

Signed-off-by: Matt Thompson <matt@boondocker.io>
---
 drivers/pci/controller/pcie-rockchip-host.c | 45 +++++++++++++++++++++++++++--
 drivers/pci/controller/pcie-rockchip.h      |  3 ++
 2 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
index 64ac6f4..d6c494f 100644
--- a/drivers/pci/controller/pcie-rockchip-host.c
+++ b/drivers/pci/controller/pcie-rockchip-host.c
@@ -19,6 +19,7 @@
 #include <linux/irq.h>
 #include <linux/irqchip/chained_irq.h>
 #include <linux/irqdomain.h>
+#include <linux/jiffies.h>
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/of_pci.h>
@@ -123,11 +124,28 @@ static int rockchip_pcie_wr_own_conf(struct rockchip_pcie *rockchip,
 				     int where, int size, u32 val)
 {
 	u32 mask, tmp, offset;
+	bool link_control = where == 0xc0 + PCI_EXP_LNKCTL;
 	void __iomem *addr;
 
 	offset = where & ~0x3;
 	addr = rockchip->apb_base + PCIE_RC_CONFIG_NORMAL_BASE + offset;
 
+	/*
+	 * Let the PHY finish initial training after Link Disable is cleared.
+	 * Retain a request made during this phase in case the data link does
+	 * not finish initializing on its own.
+	 */
+	if (link_control && (val & PCI_EXP_LNKCTL_LD)) {
+		rockchip->retrain_pending = false;
+	} else if (link_control && (val & PCI_EXP_LNKCTL_RL)) {
+		rockchip->retrain_pending = false;
+		if (!PCIE_LINK_TRAINING_DONE(rockchip_pcie_read(rockchip, PCIE_CORE_CTRL))) {
+			rockchip->retrain_pending = true;
+			rockchip->retrain_after = jiffies + msecs_to_jiffies(50);
+			val &= ~PCI_EXP_LNKCTL_RL;
+		}
+	}
+
 	if (size == 4) {
 		writel(val, addr);
 		return PCIBIOS_SUCCESSFUL;
@@ -149,8 +167,29 @@ static int rockchip_pcie_wr_own_conf(struct rockchip_pcie *rockchip,
 
 static bool rockchip_pcie_downstream_ready(struct rockchip_pcie *rockchip)
 {
-	return PCIE_LINK_UP(rockchip_pcie_read(rockchip, PCIE_CLIENT_BASIC_STATUS1)) &&
-	       PCIE_LINK_TRAINING_DONE(rockchip_pcie_read(rockchip, PCIE_CORE_CTRL));
+	u32 status = rockchip_pcie_read(rockchip, PCIE_CLIENT_BASIC_STATUS1);
+	u32 core = rockchip_pcie_read(rockchip, PCIE_CORE_CTRL);
+	u32 control;
+
+	if (!PCIE_LINK_TRAINING_DONE(core))
+		return false;
+	if (PCIE_LINK_UP(status)) {
+		rockchip->retrain_pending = false;
+		return true;
+	}
+
+	if (rockchip->retrain_pending &&
+	    time_after_eq(jiffies, rockchip->retrain_after) &&
+	    (status & PCIE_CLIENT_LINK_STATUS_MASK) == PCIE_CLIENT_LINK_STATUS_DL_INIT) {
+		rockchip->retrain_pending = false;
+		rockchip_pcie_rd_own_conf(rockchip, 0xc0 + PCI_EXP_LNKCTL, 2, &control);
+		if (!(control & PCI_EXP_LNKCTL_LD)) {
+			rockchip_pcie_wr_own_conf(rockchip, 0xc0 + PCI_EXP_LNKCTL, 2,
+						  control | PCI_EXP_LNKCTL_RL);
+		}
+	}
+
+	return false;
 }
 
 #ifdef CONFIG_ARM64
@@ -914,6 +953,8 @@ static int rockchip_pcie_suspend_noirq(struct device *dev)
 	struct rockchip_pcie *rockchip = dev_get_drvdata(dev);
 	int ret;
 
+	rockchip->retrain_pending = false;
+
 	/* disable core and cli int since we don't need to ack PME_ACK */
 	rockchip_pcie_write(rockchip, (PCIE_CLIENT_INT_CLI << 16) |
 			    PCIE_CLIENT_INT_CLI, PCIE_CLIENT_INT_MASK);
diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
index 1cc3621..5fdd06b 100644
--- a/drivers/pci/controller/pcie-rockchip.h
+++ b/drivers/pci/controller/pcie-rockchip.h
@@ -58,6 +58,7 @@
 #define   PCIE_CLIENT_NEG_LINK_WIDTH_SHIFT	6
 #define   PCIE_CLIENT_NEG_LINK_SPEED		BIT(5)
 #define PCIE_CLIENT_BASIC_STATUS1	(PCIE_CLIENT_BASE + 0x48)
+#define   PCIE_CLIENT_LINK_STATUS_DL_INIT	0x00200000
 #define   PCIE_CLIENT_LINK_STATUS_UP		0x00300000
 #define   PCIE_CLIENT_LINK_STATUS_MASK		0x00300000
 #define PCIE_CLIENT_INT_MASK		(PCIE_CLIENT_BASE + 0x4c)
@@ -325,6 +326,8 @@ struct rockchip_pcie {
 	void    __iomem *msg_region;
 	phys_addr_t msg_bus_addr;
 	bool is_rc;
+	bool retrain_pending;
+	unsigned long retrain_after;
 	struct resource *mem_res;
 };
 

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH RFC v3 6/6] PCI: rockchip-host: Mask the client correctable error interrupt
  2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
                   ` (4 preceding siblings ...)
  2026-11-16  9:35 ` [PATCH RFC v3 5/6] PCI: rockchip-host: Defer retraining during initial link training Matt Thompson via B4 Relay
@ 2026-11-16  9:35 ` Matt Thompson via B4 Relay
  2026-11-16  9:54   ` sashiko-bot
  5 siblings, 1 reply; 13+ messages in thread
From: Matt Thompson via B4 Relay @ 2026-11-16  9:35 UTC (permalink / raw)
  To: Shawn Lin, Bjorn Helgaas, Lorenzo Pieralisi,
	Krzysztof Wilczyński, Manivannan Sadhasivam, Heiko Stuebner,
	Catalin Marinas, Will Deacon
  Cc: Rob Herring, Mark Rutland, linux-pci, linux-rockchip,
	linux-arm-kernel, linux-kernel, Igor Velkov

From: Matt Thompson <matt@boondocker.io>

On RockPro64 the PCIe client correctable-error bit stayed asserted
during link bring-up despite acknowledgment. Its level interrupt
fired at roughly one million interrupts per second on CPU 0 and
starved RCU.

Mask the controller-local notification, which the driver only uses
for a debug message. Standard PCIe AER uses root port status and its
separate interrupt.

Signed-off-by: Matt Thompson <matt@boondocker.io>
---
 drivers/pci/controller/pcie-rockchip-host.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
index d6c494f..a8c2575 100644
--- a/drivers/pci/controller/pcie-rockchip-host.c
+++ b/drivers/pci/controller/pcie-rockchip-host.c
@@ -746,8 +746,14 @@ err_out:
 
 static void rockchip_pcie_enable_interrupts(struct rockchip_pcie *rockchip)
 {
-	rockchip_pcie_write(rockchip, (PCIE_CLIENT_INT_CLI << 16) &
-			    (~PCIE_CLIENT_INT_CLI), PCIE_CLIENT_INT_MASK);
+	/*
+	 * Leave the client correctable-error notification masked. On RK3399
+	 * it can remain asserted after acknowledgment during link bring-up
+	 * and monopolize the CPU. PCIe AER uses the root port's own status
+	 * and interrupt, which are independent of this client mask.
+	 */
+	rockchip_pcie_write(rockchip, (PCIE_CLIENT_INT_CLI << 16) |
+			    PCIE_CLIENT_INT_CORR_ERR, PCIE_CLIENT_INT_MASK);
 	rockchip_pcie_write(rockchip, (u32)(~PCIE_CORE_INT),
 			    PCIE_CORE_INT_MASK);
 

-- 
Git-155)



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down
  2026-11-16  9:35 ` [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down Matt Thompson via B4 Relay
@ 2026-11-16  9:46   ` sashiko-bot
  0 siblings, 0 replies; 13+ messages in thread
From: sashiko-bot @ 2026-11-16  9:46 UTC (permalink / raw)
  To: Matt Thompson; +Cc: linux-pci

> From: Matt Thompson <matt@boondocker.io>
> 
> On a RockPro64, a PCI bus rescan with the downstream FPGA unavailable
> faulted in rockchip_pcie_rd_conf() during a vendor ID read. The rescan
> task exited with interrupts disabled and the watchdog reported a lockup.
> 
> Check the controller's link-up and link-training status before issuing
> downstream config requests. Return PCIBIOS_DEVICE_NOT_FOUND while the
> link is unavailable. Root port config access still uses the local
> controller registers.
> 
> This check cannot prevent the link from dropping during the transaction.
> 
> Fixes: e77f847df54c ("PCI: rockchip: Add Rockchip PCIe controller support")
> Signed-off-by: Matt Thompson <matt@boondocker.io>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116-rfc-b4-v3-0-0bf052b68f2a@boondocker.io?part=1


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC v3 2/6] arm64: Allow synchronous abort fixups for guarded MMIO reads
  2026-11-16  9:35 ` [PATCH RFC v3 2/6] arm64: Allow synchronous abort fixups for guarded MMIO reads Matt Thompson via B4 Relay
@ 2026-11-16  9:47   ` sashiko-bot
  0 siblings, 0 replies; 13+ messages in thread
From: sashiko-bot @ 2026-11-16  9:47 UTC (permalink / raw)
  To: Matt Thompson; +Cc: linux-pci

> From: Matt Thompson <matt@boondocker.io>
> 
> An RK3399 PCI configuration read can raise a synchronous external data
> abort if it does not get a usable response. Allow an explicitly annotated
> MMIO load to return an error in this case.
> 
> Match only a current-EL synchronous external abort on a read instruction
> with the new exception table type. Keep existing handling for other
> faults, including asynchronous SError and translation table walk aborts.
> 
> Signed-off-by: Matt Thompson <matt@boondocker.io>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116-rfc-b4-v3-0-0bf052b68f2a@boondocker.io?part=2


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC v3 5/6] PCI: rockchip-host: Defer retraining during initial link training
  2026-11-16  9:35 ` [PATCH RFC v3 5/6] PCI: rockchip-host: Defer retraining during initial link training Matt Thompson via B4 Relay
@ 2026-11-16  9:51   ` sashiko-bot
  0 siblings, 0 replies; 13+ messages in thread
From: sashiko-bot @ 2026-11-16  9:51 UTC (permalink / raw)
  To: Matt Thompson; +Cc: linux-pci

> From: Matt Thompson <matt@boondocker.io>
> 
> Clearing Link Disable starts initial training. A retrain request can
> arrive while the PHY is still in Detect, when it cannot recover a data
> link that later stalls in initialization. On RockPro64 the link could
> reach L0 while the data link remained unable to pass configuration
> traffic.
> 
> Remember an early request and give initialization 50 ms to finish.
> If a later downstream access finds the PHY ready with data link
> initialization still pending, issue the saved request once and leave
> configuration access blocked until the data link becomes ready.
> 
> Signed-off-by: Matt Thompson <matt@boondocker.io>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116-rfc-b4-v3-0-0bf052b68f2a@boondocker.io?part=5


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC v3 3/6] PCI: rockchip-host: Recover downstream configuration access
  2026-11-16  9:35 ` [PATCH RFC v3 3/6] PCI: rockchip-host: Recover downstream configuration access Matt Thompson via B4 Relay
@ 2026-11-16  9:52   ` sashiko-bot
  0 siblings, 0 replies; 13+ messages in thread
From: sashiko-bot @ 2026-11-16  9:52 UTC (permalink / raw)
  To: Matt Thompson; +Cc: linux-pci

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] PCI: rockchip-host: rockchip_pcie_cfg_load() misses SError on
  Cortex-A72
--

commit f728b48c5b51b4d3ce6fd3c2e945b72fecef9c1b
Author: Matt Thompson <matt@boondocker.io>

PCI: rockchip-host: Recover downstream configuration access

This commit adds arm64 exception-table recovery for downstream PCI
configuration loads and clears the controller's AXI link-down latch before
downstream configuration reads and writes.

Link: https://lore.kernel.org/linux-pci/20261024-rockchip-pcie-resume-v1-2-58e54a5cf5f1@iav.lv/

> diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
> index 35105c129de1c..64ac6f42532d9 100644
> --- a/drivers/pci/controller/pcie-rockchip-host.c
> +++ b/drivers/pci/controller/pcie-rockchip-host.c

[ ... ]

> @@ -149,16 +153,36 @@ static bool rockchip_pcie_downstream_ready(struct rockchip_pcie *rockchip)
>  	       PCIE_LINK_TRAINING_DONE(rockchip_pcie_read(rockchip, PCIE_CORE_CTRL));
>  }
>  
> +#ifdef CONFIG_ARM64
> +/*
> + * An absent completion can raise a synchronous external abort even when
> + * the link status indicated up immediately before the config transaction.
> + */
> +#define rockchip_pcie_cfg_load(insn, addr, val)			\
> +	({								\
> +		int err = 0;					\
> +		asm volatile("1: " #insn " %w0, [%2]\n"			\
> +			     "2:\n"					\
> +			     _ASM_EXTABLE_MMIO_READ_ERR_ZERO(1b, 2b,	\
> +							    %w1, %w0)	\
> +			     : "=&r" (val), "+r" (err)			\
> +			     : "r" (addr) : "memory");			\
> +		err;							\
> +	})
> +#endif
> +
>  static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
>  				       struct pci_bus *bus, u32 devfn,
>  				       int where, int size, u32 *val)
>  {
>  	void __iomem *addr;
> +#ifdef CONFIG_ARM64
> +	u32 data;
> +	int err;
> +#endif
>  
> -	if (!rockchip_pcie_downstream_ready(rockchip)) {
> -		*val = ~0U;
> -		return PCIBIOS_DEVICE_NOT_FOUND;
> -	}
> +	if (!rockchip_pcie_downstream_ready(rockchip))
> +		goto absent;
>  
>  	addr = rockchip->reg_base + PCIE_ECAM_OFFSET(bus->number, devfn, where);
>  
> @@ -174,6 +198,28 @@ static int rockchip_pcie_rd_other_conf(struct rockchip_pcie *rockchip,
>  		rockchip_pcie_cfg_configuration_accesses(rockchip,
>  						AXI_WRAPPER_TYPE1_CFG);
>  
> +	/* Clear the AXI link-down latch before downstream config traffic. */
> +	rockchip_pcie_write(rockchip, 0, PCIE_CORE_AXI_LINK_DOWN);
> +
> +#ifdef CONFIG_ARM64
> +	if (size == 4) {
> +		err = rockchip_pcie_cfg_load(ldr, addr, data);
> +		*val = le32_to_cpu((__force __le32)data);
> +	} else if (size == 2) {
> +		err = rockchip_pcie_cfg_load(ldrh, addr, data);
> +		*val = le16_to_cpu((__force __le16)data);
> +	} else if (size == 1) {
> +		err = rockchip_pcie_cfg_load(ldrb, addr, data);
> +		*val = data;
> +	} else {
> +		*val = 0;
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +	}
> +	if (err)
> +		goto absent;
> +	/* Preserve the I/O read ordering of readl/readw/readb. */
> +	__io_ar(*val);

[Severity: High]
As reported by Igor Velkov on RFC v2 [1], does rockchip_pcie_cfg_load()
still leave Cortex-A72 cores exposed to an unrecoverable SError panic when
a downstream configuration read fails to complete?

This driver targets RK3399, which pairs Cortex-A53 cores with Cortex-A72
cores.  While Cortex-A53 reports AXI slave errors on configuration loads
as synchronous external aborts that _ASM_EXTABLE_MMIO_READ_ERR_ZERO()
catches via fixup_sea_exception(), Cortex-A72 reports them asynchronously
as SError interrupts.  When rockchip_pcie_rd_other_conf() runs on a
Cortex-A72 core and the link drops after rockchip_pcie_downstream_ready()
or fails to return a usable completion, the load instruction retires with
err == 0 and the asynchronous SError fires at the barrier in __io_ar(*val),
panicking the system in do_serror().

[1] https://lore.kernel.org/r/20261116-rockchip-host-rfc-v2-test-1@iav.lv

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116-rfc-b4-v3-0-0bf052b68f2a@boondocker.io?part=3

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC v3 6/6] PCI: rockchip-host: Mask the client correctable error interrupt
  2026-11-16  9:35 ` [PATCH RFC v3 6/6] PCI: rockchip-host: Mask the client correctable error interrupt Matt Thompson via B4 Relay
@ 2026-11-16  9:54   ` sashiko-bot
  0 siblings, 0 replies; 13+ messages in thread
From: sashiko-bot @ 2026-11-16  9:54 UTC (permalink / raw)
  To: Matt Thompson; +Cc: linux-pci

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] PCI: rockchip-host: missing Fixes trailer on bug-fix commit
--

commit f0d8b30ebd346074d58b3aee793327c70a8878bd
Author: Matt Thompson <matt@boondocker.io>

> PCI: rockchip-host: Mask the client correctable error interrupt
>
> On RockPro64 the PCIe client correctable-error bit stayed asserted
> during link bring-up despite acknowledgment. Its level interrupt
> fired at roughly one million interrupts per second on CPU 0 and
> starved RCU.
>
> Mask the controller-local notification, which the driver only uses
> for a debug message. Standard PCIe AER uses root port status and its
> separate interrupt.
>
> Signed-off-by: Matt Thompson <matt@boondocker.io>

[Severity: Low]
This fixes an interrupt storm when unmasking client interrupts in
rockchip_pcie_enable_interrupts(). Should this include:

  Fixes: e77f847df54c ("PCI: rockchip: Add Rockchip PCIe controller support")

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116-rfc-b4-v3-0-0bf052b68f2a@boondocker.io?part=6

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC v3 4/6] PCI: Resume downstream ports before sysfs Link Control writes
  2026-11-16  9:35 ` [PATCH RFC v3 4/6] PCI: Resume downstream ports before sysfs Link Control writes Matt Thompson via B4 Relay
@ 2026-11-16 10:04   ` sashiko-bot
  0 siblings, 0 replies; 13+ messages in thread
From: sashiko-bot @ 2026-11-16 10:04 UTC (permalink / raw)
  To: Matt Thompson; +Cc: linux-pci

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] PCI: sysfs: race between lockless pm_runtime_suspended() and
  RPM_RESUMING in pci_write_config()
- [Medium] PCI: incomplete runtime resume scope for config writes in
  pci_write_config()
--

commit 98319c505535590b91bff76fabaefdd4333fe854
Author: Matt Thompson <matt@boondocker.io>

PCI: Resume downstream ports before sysfs Link Control writes

This commit resumes runtime-suspended PCIe downstream ports in
pci_write_config() before writing bytes that overlap PCI_EXP_LNKCTL so a
subsequent runtime resume does not overwrite the new Link Control value
with saved state.

> diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
> index 6669e9ad5bd3b..ff47ea1f90433 100644
> --- a/drivers/pci/pci-sysfs.c
> +++ b/drivers/pci/pci-sysfs.c
> @@ -786,6 +786,7 @@ static ssize_t pci_write_config(struct file *filp, struct kobject *kobj,
>  	struct pci_dev *dev = to_pci_dev(kobj_to_dev(kobj));
>  	unsigned int size = count;
>  	loff_t init_off = off;
> +	loff_t lnkctl = pci_pcie_cap(dev) + PCI_EXP_LNKCTL;
>  	u8 *data = (u8 *) buf;
>  	int ret;
>  
> @@ -809,6 +810,21 @@ static ssize_t pci_write_config(struct file *filp, struct kobject *kobj,
>  
>  	pci_config_pm_runtime_get(dev);
>  
> +	/*
> +	 * Link Control can be written while config space remains accessible in
> +	 * runtime suspend. Restore saved state first so a later resume cannot
> +	 * replace the new link setting with the saved value.
> +	 */
> +	if (count && pci_is_pcie(dev) && pcie_downstream_port(dev) &&
> +	    pm_runtime_suspended(&dev->dev) &&

[Severity: Medium]
Can a concurrent runtime resume cause pm_runtime_suspended() to return
false before saved PCIe state has been restored?

When the port is runtime-suspended outside D3cold,
pci_config_pm_runtime_get() increments usage_count via
pm_runtime_get_noresume() and returns with dev->power.runtime_status still
at RPM_SUSPENDED.  If another thread begins resuming the port at the same
time, rpm_resume() sets runtime_status to RPM_RESUMING and drops
dev->power.lock while running pci_pm_runtime_resume():

CPU0                                  CPU1
pci_scan_bridge_extend()              pci_write_config()
  pm_runtime_get_sync()                 pci_config_pm_runtime_get()
    rpm_resume()
      runtime_status = RPM_RESUMING
      rpm_callback()
                                        pm_runtime_suspended() -> false
                                        pci_user_write_config_word(LNKCTL)
        pci_pm_runtime_resume()
          ...
            pci_restore_pcie_state()
              pcie_capability_write_word(LNKCTL)  <-- overwrites new value

Because pm_runtime_suspended() only matches RPM_SUSPENDED, CPU1 skips
pm_runtime_resume() instead of waiting in rpm_resume() for the in-flight
transition to finish, and pci_restore_pcie_state() overwrites the new
PCI_EXP_LNKCTL value with the stale pre-suspend value.  Checking
!pm_runtime_active(&dev->dev) (or calling pm_runtime_resume()
unconditionally) would wait for an in-flight resume to complete.

> +	    off < lnkctl + sizeof(u16) && off + count > lnkctl) {
> +		ret = pm_runtime_resume(&dev->dev);
> +		if (ret < 0) {
> +			pci_config_pm_runtime_put(dev);
> +			return ret;
> +		}
> +	}

[Severity: Medium]
Does restricting the runtime resume to PCI_EXP_LNKCTL on downstream ports
leave other saved registers vulnerable to being overwritten on resume?

When a PCI device runtime-suspends, pci_save_state() saves the standard
config header and PCIe capability registers including PCI_EXP_DEVCTL,
PCI_EXP_SLTCTL, PCI_EXP_RTCTL, PCI_EXP_DEVCTL2, PCI_EXP_LNKCTL2, and
PCI_EXP_SLTCTL2, all of which pci_restore_pcie_state() writes back on
resume.  If userspace writes PCI_EXP_LNKCTL2 (for example, to set Target
Link Speed) while the port is runtime-suspended and then writes
PCI_EXP_LNKCTL to retrain or enable the link, the second write triggers
pm_runtime_resume() -> pci_restore_pcie_state() and clobbers the
PCI_EXP_LNKCTL2 change with the pre-suspend value.  Writes to other saved
registers followed by a rescan, or config writes through
proc_bus_pci_write(), have the same issue because pci_config_pm_runtime_get()
only resumes devices in PCI_D3cold.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116-rfc-b4-v3-0-0bf052b68f2a@boondocker.io?part=4

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2026-11-16 10:04 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16  9:35 [PATCH RFC v3 0/6] PCI: rockchip-host: Handle unavailable downstream config space Matt Thompson via B4 Relay
2026-11-16  9:35 ` [PATCH RFC v3 1/6] PCI: rockchip-host: Skip downstream config access while link is down Matt Thompson via B4 Relay
2026-11-16  9:46   ` sashiko-bot
2026-11-16  9:35 ` [PATCH RFC v3 2/6] arm64: Allow synchronous abort fixups for guarded MMIO reads Matt Thompson via B4 Relay
2026-11-16  9:47   ` sashiko-bot
2026-11-16  9:35 ` [PATCH RFC v3 3/6] PCI: rockchip-host: Recover downstream configuration access Matt Thompson via B4 Relay
2026-11-16  9:52   ` sashiko-bot
2026-11-16  9:35 ` [PATCH RFC v3 4/6] PCI: Resume downstream ports before sysfs Link Control writes Matt Thompson via B4 Relay
2026-11-16 10:04   ` sashiko-bot
2026-11-16  9:35 ` [PATCH RFC v3 5/6] PCI: rockchip-host: Defer retraining during initial link training Matt Thompson via B4 Relay
2026-11-16  9:51   ` sashiko-bot
2026-11-16  9:35 ` [PATCH RFC v3 6/6] PCI: rockchip-host: Mask the client correctable error interrupt Matt Thompson via B4 Relay
2026-11-16  9:54   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox