Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is down

[Francesco Dolcini <francesco.dolcini@toradex.com>]

On Thu, Jun 09, 2022 at 06:17:46AM +0000, Hongxing Zhu wrote:
> > -----Original Message-----
> > From: Lucas Stach <l.stach@pengutronix.de>
> > Sent: 2022年6月8日 15:35
> > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com;
> > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com;
> > jingoohan1@gmail.com; festevam@gmail.com;
> > francesco.dolcini@toradex.com
> > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx
> > <linux-imx@nxp.com>
> > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is
> > down
> > 
> > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu:
> > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption as
> > > much as possible by disabling clocks and regulators and returning
> > > error when the link is down.
> > >
> > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
> > > ---
> > >  drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++----
> > >  1 file changed, 15 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c
> > > b/drivers/pci/controller/dwc/pci-imx6.c
> > > index 3ce3993d5797..d122c12193a6 100644
> > > --- a/drivers/pci/controller/dwc/pci-imx6.c
> > > +++ b/drivers/pci/controller/dwc/pci-imx6.c
> > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie *pci)
> > >  	/* Start LTSSM. */
> > >  	imx6_pcie_ltssm_enable(dev);
> > >
> > > -	dw_pcie_wait_for_link(pci);
> > > +	ret = dw_pcie_wait_for_link(pci);
> > > +	if (ret)
> > > +		goto err_out;
> > 
> > This adds back error handling that has been intentionally removed in
> > f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link() fails").
> > While I agree that disabling the clocks and regulators is the right thing to do
> > when we don't manage to get a link, we should still allow the driver to probe,
> > so please add a "ret = 0" to this newly added non-fatal error paths.
> > 
> Thanks for your review comments.
> There would be a long latency if the link is down and probe is finished
>  successfully.
> Since the dw_pcie_wait_for_link() would be invoked twice in every driver probe
>  and resume operation later. Each dw_pcie_wait_for_link() would consume about
>  90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency would
>  bring bad user experience.
> 
> Here are the logs when probe is allowed when PCIe link is down:
> [   55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000
> ...
> [   56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up
> [   57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up
> ...
> [   57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2136334 usecs
> 
> [   57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000
> ...
> [   58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up
> [   59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up
> ...
> [   59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2135949 usecs
> 
> So, I'm prefer that it's better to let the probe failed when link is down. 
> How do you think about that?

I think that recently Bjorn mentioned some concern with this approach,
and I agree with him.
I think that the probe of the PCIe root port should not fail if the link
is down.

What is the reason for such a long wait in dw_pcie_wait_for_link()? Is
this slowing down the resume process as a whole? Why called twice? (I'm
not familiar with that part of the code)

Francesco