Re: Question: PCIe DPC not allowing for link retraining and bus re-scan

[Keith Busch <keith.busch@intel.com>]

On Mon, Jan 30, 2017 at 11:56:23AM +0000, Gabriele Paoloni wrote:
> Hi Keith
> 
> I am looking at the current DPC implementation in:
> http://lxr.free-electrons.com/source/drivers/pci/pcie/pcie-dpc.c
> 
> 
> If my understanding is correct upon an incoming DPC event
> the DPC IRQ handler will:
> 1) stop and remove all the devices in the hierarchy under the
> downstream port that raised the event
> 2) wait for the data link to be inactive
> 3) keep the downstream port status in DPC (in fact we set the DPC
>    trigger status again here:
>    http://lxr.free-electrons.com/source/drivers/pci/pcie/pcie-dpc.c#L55)
> 
> Now looking at the specs we have (section 6.2.10):
> "After software releases the Downstream Port from DPC, the associated Link
> will normally attempt to retrain. Software can use Data Link Layer State
> Changed interrupts, DL_Active ERR_COR signaling, or both, to signal when 
> the Link reaches the DL_Active state again"
> 
> So if my understanding is correct in the current Linux implementation
> It is not possible to recover a PCIe hierarchy from a DPC event, is it correct?
> If this is correct, shouldn't we change the current implementation to release
> the port from DPC and re-scan the secondary bus after the conditions described
> in "PCIe 3.1 section 6.2.10" are met (I.e. <<Data Link Layer Link Active bit in
> the 15 Link Status register reads 0b>> and if it is a root port also <<until
> the DPC RP Busy bit reads 0b>>) ?

That sounds reasonable. In all testing I've done, DPC events were
triggered as part of a hot-plug event, so the secondary bus rescan would
automatically happen when a device is hot inserted. The DPC driver didn't
need to initiate the rescan in that case.

But surprise removal is certainly not the only condition DPC can contain a
port, so I agree with you that we may require pcie-dpc initiate a rescan
after releasing the port from containment. I also agree on needing to
poll RP busy for root ports.

Do you want to send patches for these? I could also write them up,
but I don't have a DPC capable root port to test that part.

Thanks,
Keith