Re: [Bug 219513] New: PCIe drivers do not bind

Re: [Bug 219513] New: PCIe drivers do not bind

[Bjorn Helgaas]

On Tue, Nov 19, 2024 at 12:55:36PM +0000, bugzilla-daemon@kernel.org wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=219513
> 
>           Hardware: Sparc64
>           Priority: P3
>           Reporter: dullfire@yahoo.com
>         Regression: No
> 
> Created attachment 307241
>   --> https://bugzilla.kernel.org/attachment.cgi?id=307241&action=edit
> debug info (some shell commands to check the PCIe devices and drivers)
> 
> In linux-next (next-20241118), since
> commit 03cfe0e05650 ("PCI/pwrctl: Ensure that the pwrctl drivers are probed
> before the PCI client drivers")
> PCIe drivers no longer bind (at least on the tested SPARCv9 system).
> 
> It appears a "supplier" devlink is created, however it is are dormant. see
> attached "bug-info.txt"

Thanks for the report.  It sounds like you bisected this to
03cfe0e05650?  Can you attach a complete dmesg log to the bugzilla?

This commit is queued for v6.13, and the merge window is now open, so
if it's a regression, we need to resolve it or drop it ASAP.

Re: [Bug 219513] New: PCIe drivers do not bind

[Bartosz Golaszewski]

On Tue, 19 Nov 2024 at 15:04, Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Tue, Nov 19, 2024 at 12:55:36PM +0000, bugzilla-daemon@kernel.org wrote:
> > https://bugzilla.kernel.org/show_bug.cgi?id=219513
> >
> >           Hardware: Sparc64
> >           Priority: P3
> >           Reporter: dullfire@yahoo.com
> >         Regression: No
> >
> > Created attachment 307241
> >   --> https://bugzilla.kernel.org/attachment.cgi?id=307241&action=edit
> > debug info (some shell commands to check the PCIe devices and drivers)
> >
> > In linux-next (next-20241118), since
> > commit 03cfe0e05650 ("PCI/pwrctl: Ensure that the pwrctl drivers are probed
> > before the PCI client drivers")
> > PCIe drivers no longer bind (at least on the tested SPARCv9 system).
> >
> > It appears a "supplier" devlink is created, however it is are dormant. see
> > attached "bug-info.txt"
>
> Thanks for the report.  It sounds like you bisected this to
> 03cfe0e05650?  Can you attach a complete dmesg log to the bugzilla?
>
> This commit is queued for v6.13, and the merge window is now open, so
> if it's a regression, we need to resolve it or drop it ASAP.

Dullfire: is the DTS for this platform publicly available? If not, can
you at least post the PCI host controller and all its children nodes
here?

Bart

Re: [Bug 219513] New: PCIe drivers do not bind

[Dullfire]

On 11/19/24 08:16, Bartosz Golaszewski wrote:
> On Tue, 19 Nov 2024 at 15:04, Bjorn Helgaas <helgaas@kernel.org> wrote:
>>
>> On Tue, Nov 19, 2024 at 12:55:36PM +0000, bugzilla-daemon@kernel.org wrote:
>>> https://bugzilla.kernel.org/show_bug.cgi?id=219513
>>>
>>>           Hardware: Sparc64
>>>           Priority: P3
>>>           Reporter: dullfire@yahoo.com
>>>         Regression: No
>>>
>>> Created attachment 307241
>>>   --> https://bugzilla.kernel.org/attachment.cgi?id=307241&action=edit
>>> debug info (some shell commands to check the PCIe devices and drivers)
>>>
>>> In linux-next (next-20241118), since
>>> commit 03cfe0e05650 ("PCI/pwrctl: Ensure that the pwrctl drivers are probed
>>> before the PCI client drivers")
>>> PCIe drivers no longer bind (at least on the tested SPARCv9 system).
>>>
>>> It appears a "supplier" devlink is created, however it is are dormant. see
>>> attached "bug-info.txt"
>>
>> Thanks for the report.  It sounds like you bisected this to
>> 03cfe0e05650?  Can you attach a complete dmesg log to the bugzilla?
>>
>> This commit is queued for v6.13, and the merge window is now open, so
>> if it's a regression, we need to resolve it or drop it ASAP.
> 
> Dullfire: is the DTS for this platform publicly available? If not, can
> you at least post the PCI host controller and all its children nodes
> here?
> 
> Bart

Bart: I attached to the bug report a dts extracted from the system.

Just a FYI: SPARC systems (including linux) get their device tree from the
open firmware, which likely dynamically generates at least parts of it, so
there is not a discrete source for it.

Regards,
Jonathan Currier

Re: [Bug 219513] New: PCIe drivers do not bind

[Manivannan Sadhasivam]

On Tue, Nov 19, 2024 at 10:57:26AM -0600, Dullfire wrote:
> 
> 
> On 11/19/24 08:16, Bartosz Golaszewski wrote:
> > On Tue, 19 Nov 2024 at 15:04, Bjorn Helgaas <helgaas@kernel.org> wrote:
> >>
> >> On Tue, Nov 19, 2024 at 12:55:36PM +0000, bugzilla-daemon@kernel.org wrote:
> >>> https://bugzilla.kernel.org/show_bug.cgi?id=219513
> >>>
> >>>           Hardware: Sparc64
> >>>           Priority: P3
> >>>           Reporter: dullfire@yahoo.com
> >>>         Regression: No
> >>>
> >>> Created attachment 307241
> >>>   --> https://bugzilla.kernel.org/attachment.cgi?id=307241&action=edit
> >>> debug info (some shell commands to check the PCIe devices and drivers)
> >>>
> >>> In linux-next (next-20241118), since
> >>> commit 03cfe0e05650 ("PCI/pwrctl: Ensure that the pwrctl drivers are probed
> >>> before the PCI client drivers")
> >>> PCIe drivers no longer bind (at least on the tested SPARCv9 system).
> >>>
> >>> It appears a "supplier" devlink is created, however it is are dormant. see
> >>> attached "bug-info.txt"
> >>
> >> Thanks for the report.  It sounds like you bisected this to
> >> 03cfe0e05650?  Can you attach a complete dmesg log to the bugzilla?
> >>
> >> This commit is queued for v6.13, and the merge window is now open, so
> >> if it's a regression, we need to resolve it or drop it ASAP.
> > 
> > Dullfire: is the DTS for this platform publicly available? If not, can
> > you at least post the PCI host controller and all its children nodes
> > here?
> > 
> > Bart
> 
> Bart: I attached to the bug report a dts extracted from the system.
> 
> Just a FYI: SPARC systems (including linux) get their device tree from the
> open firmware, which likely dynamically generates at least parts of it, so
> there is not a discrete source for it.
> 

Thanks Jonathan for the dts drop! Looks like the platform devices for the pcie
nodes were created elsewhere for the SPARC systems. Could you please test the
below diff and see if it fixes the issue?

```
diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index 7a061cc860d5..e70f4c089cd4 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -394,7 +394,7 @@ void pci_bus_add_device(struct pci_dev *dev)
         * PCI client drivers.
         */
        pdev = of_find_device_by_node(dn);
-       if (pdev) {
+       if (pdev && of_pci_is_supply_present(dn)) {
                if (!device_link_add(&dev->dev, &pdev->dev,
                                     DL_FLAG_AUTOREMOVE_CONSUMER))
                        pci_err(dev, "failed to add device link between %s and %s\n",
```

- Mani

> Regards,
> Jonathan Currier

-- 
மணிவண்ணன் சதாசிவம்

Re: [Bug 219513] New: PCIe drivers do not bind

[Dullfire]

On 11/19/24 11:27, Manivannan Sadhasivam wrote:
> On Tue, Nov 19, 2024 at 10:57:26AM -0600, Dullfire wrote:
>>
>> Bart: I attached to the bug report a dts extracted from the system.
>>
>> Just a FYI: SPARC systems (including linux) get their device tree from the
>> open firmware, which likely dynamically generates at least parts of it, so
>> there is not a discrete source for it.
>>
> 
> Thanks Jonathan for the dts drop! Looks like the platform devices for the pcie
> nodes were created elsewhere for the SPARC systems. Could you please test the
> below diff and see if it fixes the issue?
> 
> ```
> diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
> index 7a061cc860d5..e70f4c089cd4 100644
> --- a/drivers/pci/bus.c
> +++ b/drivers/pci/bus.c
> @@ -394,7 +394,7 @@ void pci_bus_add_device(struct pci_dev *dev)
>          * PCI client drivers.
>          */
>         pdev = of_find_device_by_node(dn);
> -       if (pdev) {
> +       if (pdev && of_pci_is_supply_present(dn)) {
>                 if (!device_link_add(&dev->dev, &pdev->dev,
>                                      DL_FLAG_AUTOREMOVE_CONSUMER))
>                         pci_err(dev, "failed to add device link between %s and %s\n",
> ```
> 
> - Mani


Mani: Yup, that patch does the trick in my test case. The PCIe drivers all bound
and function.


Regards,
Jonathan Currier

Re: [Bug 219513] New: PCIe drivers do not bind

[Dullfire]

On 11/19/24 08:04, Bjorn Helgaas wrote:
> On Tue, Nov 19, 2024 at 12:55:36PM +0000, bugzilla-daemon@kernel.org wrote:
>> https://bugzilla.kernel.org/show_bug.cgi?id=219513
>>
>>           Hardware: Sparc64
>>           Priority: P3
>>           Reporter: dullfire@yahoo.com
>>         Regression: No
>>
>> Created attachment 307241
>>   --> https://bugzilla.kernel.org/attachment.cgi?id=307241&action=edit
>> debug info (some shell commands to check the PCIe devices and drivers)
>>
>> In linux-next (next-20241118), since
>> commit 03cfe0e05650 ("PCI/pwrctl: Ensure that the pwrctl drivers are probed
>> before the PCI client drivers")
>> PCIe drivers no longer bind (at least on the tested SPARCv9 system).
>>
>> It appears a "supplier" devlink is created, however it is are dormant. see
>> attached "bug-info.txt"
> 
> Thanks for the report.  It sounds like you bisected this to
> 03cfe0e05650?  Can you attach a complete dmesg log to the bugzilla?
Yes, a bisect pointed at 03cfe0e05650. After finding that I tried the patch below
to verify, since a trivial revert wasn't possible on next-20241118, and the PCIe
drivers started binding again, and everything seemed functional.

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index b0e3cfdaa276..6d87773784b9 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -404,7 +404,7 @@ void pci_bus_add_device(struct pci_dev *dev)
         * PCI client drivers.
         */
        pdev = of_find_device_by_node(dn);
-       if (pdev) {
+       if (pdev && false) {
                if (!device_link_add(&dev->dev, &pdev->dev,
                                     DL_FLAG_AUTOREMOVE_CONSUMER))
                        pci_err(dev, "failed to add device link between %s and %s\n",

Obviously not a fix, but it did help verify the issue.
> 
> This commit is queued for v6.13, and the merge window is now open, so
> if it's a regression, we need to resolve it or drop it ASAP.

Regards,
Jonathan Currier