Re: PCIe AER generates no interrupts on host (ZynqMP)

[Bjorn Helgaas <helgaas@kernel.org>]

On Thu, Jan 13, 2022 at 08:13:55AM +0100, Stefan Roese wrote:
> On 1/12/22 18:49, Bjorn Helgaas wrote:

> > Ah.  I assume you have:
> > 
> >    00:00.0 Root Port to [bus 01-??]
> >    01:00.0 Switch Upstream Port to [bus 02-??]
> >    02:0?.0 Switch Downstream Port to [bus 04-??]
> 
> This is correct, yes.
> 
> > pcie_portdrv_probe() claims 00:00.0 and clears CERE NFERE FERE URRE.
> > 
> > aer_probe() claims 00:00.0 and enables CERE NFERE FERE URRE for all
> > downstream devices, including 01:00.0.
> > 
> > pcie_portdrv_probe() claims 01:00.0 and clears CERE NFERE FERE URRE
> > again.
> > 
> > aer_probe() declines to claim 01:00.0 because it's not a Root Port, so
> > CERE NFERE FERE URRE remain cleared.

> I'm baffled a bit, that this problem of AER reporting being disabled in
> the DevCtl regs of PCIe ports (all non root ports) was not noticed for
> this long time. As AER is practically disabled in such setups.

The more common configuration is a Root Port leading directly to an
Endpoint.  In that case, there would be no pcie_portdrv_probe() in the
middle to disable reporting after aer_probe() has enabled it.

The issue you're seeing happens because of the switch in the middle, 
which is becoming more common recently with Thunderbolt.

I poked around on my laptop (Dell 5520 running v5.4):

  00:01.0 Root Port       to [bus 01]          CorrErr-
  01:00.0 NVIDIA GPU                           CorrErr-

  00:1c.0 Root Port       to [bus 02]     AER  CorrErr+
  02:00.0 Intel NIC                       AER  CorrErr-  <-- iwlwifi

  00:1c.1 Root Port       to [bus 03]     AER  CorrErr+
  03:00.0 Realtek card reader             AER  CorrErr-  <-- rtsx_pci

  00:1d.0 Root Port       to [bus 04]     AER  CorrErr+
  04:00.0 NVMe                            AER  CorrErr+

  00:1d.6 Root Port       to [bus 06-3e]  AER  CorrErr+
  06:00.0 Thunderbolt USP to [bus 07-3e]  AER  CorrErr-
  07:00.0 Thunderbolt DSP to [bus 08]     AER  CorrErr-
  ...                                          CorrErr-

Everything in the Thunderbolt hierarchy has reporting disabled,
probably because of the issue you are pointing out.

I can't explain the iwlwifi and rtsx_pci cases.  Both devices have AER
and are directly below a Root Port that also has AER, so I would think
reporting should be enabled.

Bjorn