[PATCH v3 00/18] PCI: rcar-gen4: Recover from link down and route Root Port interrupts

[Koichiro Den <den@valinux.co.jp>]

Hi,

This series improves error handling in the pcie-rcar-gen4 driver, which
supports PCIe controllers in R-Car Gen4 SoCs and the PCIe4 controller
in R-Car Gen5 SoCs. It fixes unexpected link-down handling so the host
doesn't hang, and wires up missing Root Port interrupts (AER, PME,
bandwidth notifications) so port services actually work. I tested on
an R-Car S4 Spider (r8a779f0); Marek tested the link state fix and the
Root Port AER and bandwidth paths on an R-Car V4H Sparrow Hawk.

A few hardware quirks made this tricky:

1. PCIEINTSTS0 link-up bits don't track the actual link state. The
   driver never noticed when the link dropped and kept trying config
   accesses on the dead link. Patch 3 combines the APP link-up event
   check from Figure 104.5 with the PORT_LINK_DEBUG1 live link check,
   which patch 2 factors out of the DWC core. Startup clears the APP
   latches before enabling LTSSM, and the DWC core polls the combined
   condition on the RC side.

2. On S4, a DBI access immediately after an unexpected link down can
   hang the host. Commit 0056d29f8c1b ("PCI: rcar-gen4: Assure reset
   occurs before DBI access") describes an SError on V4H after reset
   deassertion, but whether the S4 hang shares the same underlying cause
   has not been verified.

   Adding a delay before the DBI access avoided the hang in my tests,
   but that alone would not provide link-down recovery. Also, the reset
   request shares intreq_pcim_sub with iMSI-RX, while AER arrives on
   another IRQ. Delaying AER dispatch alone would therefore leave the
   MSI handler exposed, and both paths would still need coordination
   with the controller reset.

   The driver handles the reset request itself and schedules
   pci_host_handle_link_down(), as the rockchip and qcom drivers do.
   This also provides recovery with older DTs that have no "aer"
   interrupt, without relying on AER to initiate it. The reset callback
   shares the reset sequence used at probe, including the reset-status
   readback and delay added by 0056d29f8c1b.

   This covers the interrupt paths that fire on link down. Port service
   threads or work already queued or running at that point are not
   stopped.

3. Root Port interrupts only trigger APP status bits on platform IRQs.
   The controller appears to lack SII2MSI, so Root Port MSIs never reach
   the GIC ITS. The Root Port's INTx is routed to intreq_pcim_sub as
   well, which the driver holds, so port services can't request it
   either. This series works around it by hiding Root Port MSI caps
   across the board and emulating INTx using a virtual IRQ domain, fed
   by the "aer" IRQ and intreq_pcim_sub.

Patch 1 is only loosely related: it adds Renesas to the RAS DES VSEC
list so the DWC debugfs error injection works on R-Car. I used it to
test the Root Port AER path (see below) and included it here for that
reason. Happy to send it separately if preferred.

Based on next-20261002, with Marek's PM ops v5 patch applied first:
https://lore.kernel.org/r/20261004011851.866833-1-marek.vasut+renesas@mailbox.org/

Note: backward compatibility with older DTs is kept. Without the "aer"
      interrupt, only Root Port AER remains unavailable. See the Testing
      section below.

Retesting with v3
-----------------

Setup: R-Car S4 Spider (RC) linked to another S4 Spider running the
       pci-epf-test endpoint. pci_endpoint_test is bound on the RC side.

  1. Link down / recovery. On the EP side, toggle the endpoint controller
     off and on. The short pause keeps the endpoint away long enough for
     the RC to notice, but brings it back within the reset window so
     recovery can succeed. Adopted the test approach from [1]. Make sure
     that the unused function 0000:01:00.1 was removed on the RC before
     testing.

     # cd /sys/kernel/config/pci_ep
     # echo 0 > controllers/e65d0000.pcie-ep/start
     # sleep 0.1
     # echo 1 > controllers/e65d0000.pcie-ep/start

     Expected on the RC dmesg:
       pcieport 0000:00:00.0: Recovering Root Port due to Link Down
       pcieport 0000:00:00.0: Root Port has been reset
       pcieport 0000:00:00.0: AER: device recovery successful

     and the "msi" (intreq_pcim_sub) interrupt count going up in
     /proc/interrupts. Without this series nothing shows up here: the
     link comes back on its own once the endpoint returns, but the RC
     never notices the outage and the endpoint is left unconfigured
     (see 4). Config accesses issued while the link is down hang the
     host.

     [1] https://lore.kernel.org/r/abFMa6DCGGLUHddA@fedora/

  2. Bandwidth notification. On the RC, retrain the link:

     # setpci -s 00:00.0 CAP_EXP+0x10.w=0x0c23

     Expected:
     - the virtual Root Port IRQ (rcar-gen4-rp in /proc/interrupts,
       shared by PCIe PME, aerdrv and PCIe bwctrl) fires once
     - bwctrl clears LnkSta.LBMS
       (setpci -s 00:00.0 CAP_EXP+0x12.w reads 0x2024 again).

     Before the series LnkSta read 0xe024 afterwards, LBMS and LABS
     stuck.

     This test does not generate a PME. The PME service shares the
     virtual IRQ, but its path was not exercised separately.

  3. Root Port AER. On the RC, inject an LCRC error with the DWC debugfs
     (patch 1) and issue one config read so a TLP actually goes out:

     # cd /sys/kernel/debug/dwc_pcie_e65d0000.pcie/rasdes_err_inj
     # echo 1 > rx_lcrc            # error detected by the Root Port
     # setpci -s 01:00.0 VENDOR_ID.w
     # echo 1 > tx_lcrc            # error detected by the endpoint,
     # setpci -s 01:00.0 VENDOR_ID.w    # reported back with ERR_COR

     Expected on the RC dmesg, respectively:
      pcieport 0000:00:00.0: PCIe Bus Error: severity=Correctable
      pcieport 0000:00:00.0:   [ 6] BadTLP | Receiver | Data Link Layer

      pcieport 0000:00:00.0: AER: Correctable Error message received from 0000:01:00.0
      pci-endpoint-test 0000:01:00.0: PCIe Bus Error: severity=Correctable
      pci-endpoint-test 0000:01:00.0:   [ 6] BadTLP | Receiver | Data Link Layer

     plus the virtual Root Port IRQ count and aer_rootport_total_err_cor
     going up by one each time. The link stays up throughout, the DLL
     retry recovers the TLP. Before the series nothing is reported.

  4. Regression check. Run pci_endpoint_test after step 1. PASS/FAIL/SKIP
     counts match a run without step 1.

Configurations:

  a. Without this series**
  b. GIC ITS, DT with the new "aer" interrupt (this series)
  c. GIC ITS, DT without "aer" (b43aa6a6ebe8 ("arm64: dts: renesas:
     r8a779f0: Add GICv3 ITS and update PCIe nodes") or later)
  d. iMSI-RX, DT before b43aa6a6ebe8 (no msi-parent, no "aer")

Result:
             recovery   bwctrl       RP AER    pcitest
  ---------------------------------------------------------
  a.           none         no         no      all FAIL
  b.            ok          ok         ok      no change
  c.            ok          ok         n/a*    no change
  d.            ok          ok         n/a*    no change

  *  Root Port AER needs the "aer" interrupt; without it the behaviour is
     unchanged from before the series.
  ** Only patch 1 applied on top of the base, so the same debugfs error
     injection could be used for the comparison. pci_endpoint_test fails
     across the board there because nothing restores the endpoint after
     the toggle.

Best regards,
Koichiro
---
Changes in v3:
  - Rebase onto next-20261002 with Marek's PM ops v5 applied first.
  - Keep parent IRQs registered across suspend/resume to fix the
    unbalanced enable Geert reported. Request them in probe with
    IRQF_NO_AUTOEN and enable/disable them in .init()/.deinit().
  - Use a freezable workqueue for recovery and reject suspend while
    reinitialization is pending. Re-arm Root Port events only after
    resume has set up the link.
  - Factor out the PORT_LINK_DEBUG1 check (patch 2) and reuse it in
    patch 3. (Marek)
  - Replace .reinit() with .configure, folding v2 patch 8 into patch 9.
    Drop the reset mutex and rely on PCI core serialization. (Marek)
  - Free the MSI domain after host .deinit(), so the driver can stop
    its parent IRQ first (patch 10). This closes the known gap in v2.
  - Reset the controller if recovery finds no Root Port, so a successful
    reset can restore interrupt delivery.
  - Drop the raw lock around virtual IRQ dispatch. Preserve MSI-form
    latches in irq_ack() while the IRQ is in progress or disabled,
    avoiding lost notifications without nesting port service locks.
  - Add patch 7's Fixes tag, clarify SoC generations, separate guard()
    from goto-based cleanup, and simplify IRQ return statements.
    (Marek, Sashiko)
  - Add the V4H and V4M DTS patches 17 and 18. (Marek)
  - Collect Reviewed-by (Marek, Krzysztof) and Tested-by (Marek) tags.
    Drop Marek's Reviewed-by from the reworked patches 12 and 14.

Changes in v2:
  - Rebased onto next-20260925, including Marek's R-Car X5H support.
  - Keep .link_up() and require both the APP link-up events and the
    PORT_DEBUG1 live link check. Clear the APP latches before enabling
    LTSSM and reuse the DWC core's polling on the RC side. (Marek)
  - Let the R-Car driver own intreq_pcim_sub in all configurations (new
    patch 10) and check the reset request from its own handler instead
    of hooking into the DWC chained handler. Replace the pre_msi_irq host
    op from v1 with an export of dw_handle_msi_irq() (patch 4). (Marek)
  - Keep the reset_control_status() check before asserting the power
    reset. (Marek)
  - Separate controller reinitialization from .init()/.deinit() and split
    out preparatory changes.
  - Use bool flags instead of a state bitmask. (Marek)
  - Keep Root Port AER notifications masked until .post_init, after
    enumeration. In v1, they could be enabled during port-service probing.
  - Keep APP interrupt sources masked if the Root Port reset callback
    fails, regardless of the reset trigger.
  - Hold a reference on the Root Port in the link-down recovery work.
  - Clear only the MSI-form Root Port latches through PCIEINTSTS0CLR;
    the INTx bits are reserved there.
  - Serialize dispatches to the virtual Root Port IRQ from its two
    parent interrupts.
  - Collected Marek's Reviewed-by on patches 1, 3 and 15.

v2: https://lore.kernel.org/r/20260928165230.3397664-1-den@valinux.co.jp/
v1: https://lore.kernel.org/r/20260918032038.2216471-1-den@valinux.co.jp/


Koichiro Den (18):
  PCI: dwc: Add Renesas to the RAS DES VSEC list
  PCI: dwc: Factor out the PORT_LINK_DEBUG1 link-up check
  PCI: rcar-gen4: Check live link status in link_up()
  dt-bindings: PCI: rcar-gen4: Add optional "aer" interrupt
  PCI: dwc: Export dw_handle_msi_irq()
  PCI: rcar-gen4: Move deinitialization helpers before SoC
    initialization
  PCI: rcar-gen4: Assert resets when Gen5 SoC PHY initialization fails
  PCI: rcar-gen4: Separate hardware setup from resource acquisition
  PCI: rcar-gen4: Add Root Port reset support
  PCI: dwc: Free the MSI domain after the host .deinit() callback
  PCI: rcar-gen4: Take over the iMSI-RX interrupt
  PCI: rcar-gen4: Recover the Root Port on link down
  PCI: dwc: Let glue drivers hide the Root Port MSI capabilities
  PCI: rcar-gen4: Route Root Port AER to a virtual Root Port IRQ
  PCI: rcar-gen4: Route Root Port PME and bandwidth notifications
  arm64: dts: renesas: r8a779f0: Describe the PCIe AER interrupts
  arm64: dts: renesas: r8a779g0: Describe the PCIe AER interrupts
  arm64: dts: renesas: r8a779h0: Describe the PCIe AER interrupt

 .../bindings/pci/rcar-gen4-pci-host.yaml      |  10 +-
 arch/arm64/boot/dts/renesas/r8a779f0.dtsi     |  10 +-
 arch/arm64/boot/dts/renesas/r8a779g0.dtsi     |  10 +-
 arch/arm64/boot/dts/renesas/r8a779h0.dtsi     |   5 +-
 .../pci/controller/dwc/pcie-designware-host.c |  38 +-
 drivers/pci/controller/dwc/pcie-designware.c  |  15 +-
 drivers/pci/controller/dwc/pcie-designware.h  |   2 +
 drivers/pci/controller/dwc/pcie-rcar-gen4.c   | 723 ++++++++++++++++--
 include/linux/pcie-dwc.h                      |   2 +
 9 files changed, 724 insertions(+), 91 deletions(-)


base-commit: f0406245cb9855e6318335a8a223551354291a46
prerequisite-patch-id: 5b4c9da1333342baa90cf6a86dc879e8382d3f65
-- 
2.51.0