Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Scott Wood]

On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
> Instead of trying to match and probe platform and AMBA devices right
> after each is registered, delay their probes until device_initcall_sync.
> 
> This means that devices will start probing once all built-in drivers
> have registered, and after all platform and AMBA devices from the DT
> have been registered already.
> 
> This allows us to prevent deferred probes by probing dependencies on
> demand.
> 
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> ---
> 
> Changes in v4:
> - Also defer probes of AMBA devices registered from the DT as they can
>   also request resources.
> 
>  drivers/of/platform.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)

This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF platform 
device, and it must be probed before pcibios_init() which is a 
subsys_initcall(), or else the PCI bus never gets scanned.

-Scott

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Rob Herring]

On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com> wrote:
> On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
>> Instead of trying to match and probe platform and AMBA devices right
>> after each is registered, delay their probes until device_initcall_sync.
>>
>> This means that devices will start probing once all built-in drivers
>> have registered, and after all platform and AMBA devices from the DT
>> have been registered already.
>>
>> This allows us to prevent deferred probes by probing dependencies on
>> demand.
>>
>> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>> ---
>>
>> Changes in v4:
>> - Also defer probes of AMBA devices registered from the DT as they can
>>   also request resources.
>>
>>  drivers/of/platform.c | 11 ++++++++---
>>  1 file changed, 8 insertions(+), 3 deletions(-)
>
> This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF platform
> device, and it must be probed before pcibios_init() which is a
> subsys_initcall(), or else the PCI bus never gets scanned.

Thanks for the report. This is probably getting dropped, but it could
be disabled for PPC.

Any plans to fix this and make PCI hosts hotplugable? For the scanning
part, generally the host controller drivers are responsible for
scanning their bus now.

Rob

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Scott Wood]

On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
> On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com> 
> wrote:
> > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
> > > Instead of trying to match and probe platform and AMBA devices right
> > > after each is registered, delay their probes until device_initcall_sync.
> > > 
> > > This means that devices will start probing once all built-in drivers
> > > have registered, and after all platform and AMBA devices from the DT
> > > have been registered already.
> > > 
> > > This allows us to prevent deferred probes by probing dependencies on
> > > demand.
> > > 
> > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > ---
> > > 
> > > Changes in v4:
> > > - Also defer probes of AMBA devices registered from the DT as they can
> > >   also request resources.
> > > 
> > >  drivers/of/platform.c | 11 ++++++++---
> > >  1 file changed, 8 insertions(+), 3 deletions(-)
> > 
> > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF platform
> > device, and it must be probed before pcibios_init() which is a
> > subsys_initcall(), or else the PCI bus never gets scanned.
> 
> Thanks for the report. This is probably getting dropped, but it could
> be disabled for PPC.

I don't think that adding another arbitrary arch difference would be the 
right solution.

> Any plans to fix this and make PCI hosts hotplugable? For the scanning
> part, generally the host controller drivers are responsible for
> scanning their bus now.

Scanning from the host controller driver seems like a reasonable goal, though 

it'd take a bit of digging to extract whatever other things fsl_pci may 
depend on from the common PPC PCI code, in particular the various things that 

pcibios_resource_survey() does after all PCI buses have been scanned.

There's also check_swiotlb_enabled(), another subsys_initcall, which frees 
the swiotlb memory if ppc_swiotlb_enable hasn't been set.  The PCI host 
controller probe sets ppc_swiotlb_enable if it wasn't able to create an 
inbound mapping for all RAM.  Even if we were to change that to a later 
initcall, there's nothing later than late_initcall that we could use.

-Scott

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Tomeu Vizoso]

On 22 October 2015 at 00:51, Scott Wood <scottwood@freescale.com> wrote:
> On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
>> On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com>
>> wrote:
>> > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
>> > > Instead of trying to match and probe platform and AMBA devices right
>> > > after each is registered, delay their probes until device_initcall_sync.
>> > >
>> > > This means that devices will start probing once all built-in drivers
>> > > have registered, and after all platform and AMBA devices from the DT
>> > > have been registered already.
>> > >
>> > > This allows us to prevent deferred probes by probing dependencies on
>> > > demand.
>> > >
>> > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>> > > ---
>> > >
>> > > Changes in v4:
>> > > - Also defer probes of AMBA devices registered from the DT as they can
>> > >   also request resources.
>> > >
>> > >  drivers/of/platform.c | 11 ++++++++---
>> > >  1 file changed, 8 insertions(+), 3 deletions(-)
>> >
>> > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF platform
>> > device, and it must be probed before pcibios_init() which is a
>> > subsys_initcall(), or else the PCI bus never gets scanned.
>>
>> Thanks for the report. This is probably getting dropped, but it could
>> be disabled for PPC.
>
> I don't think that adding another arbitrary arch difference would be the
> right solution.

I think Rob meant temporarily disable it while things get fixed. At
least, I don't see any reason why PPC wouldn't benefit from this
series.

Regards,

Tomeu

>> Any plans to fix this and make PCI hosts hotplugable? For the scanning
>> part, generally the host controller drivers are responsible for
>> scanning their bus now.
>
> Scanning from the host controller driver seems like a reasonable goal, though
>
> it'd take a bit of digging to extract whatever other things fsl_pci may
> depend on from the common PPC PCI code, in particular the various things that
>
> pcibios_resource_survey() does after all PCI buses have been scanned.
>
> There's also check_swiotlb_enabled(), another subsys_initcall, which frees
> the swiotlb memory if ppc_swiotlb_enable hasn't been set.  The PCI host
> controller probe sets ppc_swiotlb_enable if it wasn't able to create an
> inbound mapping for all RAM.  Even if we were to change that to a later
> initcall, there's nothing later than late_initcall that we could use.
>
> -Scott
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Scott Wood]

On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
> On 22 October 2015 at 00:51, Scott Wood <scottwood@freescale.com> wrote:
> > On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
> > > On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com>
> > > wrote:
> > > > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
> > > > > Instead of trying to match and probe platform and AMBA devices right
> > > > > after each is registered, delay their probes until 
> > > > > device_initcall_sync.
> > > > > 
> > > > > This means that devices will start probing once all built-in drivers
> > > > > have registered, and after all platform and AMBA devices from the DT
> > > > > have been registered already.
> > > > > 
> > > > > This allows us to prevent deferred probes by probing dependencies on
> > > > > demand.
> > > > > 
> > > > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > > > ---
> > > > > 
> > > > > Changes in v4:
> > > > > - Also defer probes of AMBA devices registered from the DT as they 
> > > > > can
> > > > >   also request resources.
> > > > > 
> > > > >  drivers/of/platform.c | 11 ++++++++---
> > > > >  1 file changed, 8 insertions(+), 3 deletions(-)
> > > > 
> > > > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF 
> > > > platform
> > > > device, and it must be probed before pcibios_init() which is a
> > > > subsys_initcall(), or else the PCI bus never gets scanned.
> > > 
> > > Thanks for the report. This is probably getting dropped, but it could
> > > be disabled for PPC.
> > 
> > I don't think that adding another arbitrary arch difference would be the
> > right solution.
> 
> I think Rob meant temporarily disable it while things get fixed. At
> least, 

So, what is the permanent fix for the swiotlb issue (or more generally, the 
inability to have a late_initcall that runs after non-module, non-hotplug 
platform devices have been probed)?

> I don't see any reason why PPC wouldn't benefit from this
> series.

It's not clear to me what the benefit of this is at all, much less for PPC.   
What is the fundamental problem with deferred probes?  In the cover letter 
you say this change saves 2.3 seconds, but where is that time being consumed? 
 Are the drivers taking too long in their probe function trying to initialize 
and then deferring, rather than checking for dependencies up front?  Or are 
there really so many devices and such a pessimal ordering that most of the 
time is spent iterating through and reordering the list, with each defer 
happening quickly?

Even if something different does need to be done at this level, forcing all 
OF platform devices to be probed at the late_initcall level seems quite 
intrusive.  You limited it to OF because people complained that other things 
will break.  Things still broke.  Surely there's a better way to address the 
problem.  Can't the delay be requested by drivers that might otherwise need 
to defer (which could be done incrementally, focusing on the worst 
performance problems), rather than enabling it for everything?

-Scott

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Rafael J. Wysocki]

On Thursday, October 22, 2015 04:27:10 PM Scott Wood wrote:
> On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
> > On 22 October 2015 at 00:51, Scott Wood <scottwood@freescale.com> wrote:
> > > On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
> > > > On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com>
> > > > wrote:
> > > > > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
> > > > > > Instead of trying to match and probe platform and AMBA devices right
> > > > > > after each is registered, delay their probes until 
> > > > > > device_initcall_sync.
> > > > > > 
> > > > > > This means that devices will start probing once all built-in drivers
> > > > > > have registered, and after all platform and AMBA devices from the DT
> > > > > > have been registered already.
> > > > > > 
> > > > > > This allows us to prevent deferred probes by probing dependencies on
> > > > > > demand.
> > > > > > 
> > > > > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > > > > ---
> > > > > > 
> > > > > > Changes in v4:
> > > > > > - Also defer probes of AMBA devices registered from the DT as they 
> > > > > > can
> > > > > >   also request resources.
> > > > > > 
> > > > > >  drivers/of/platform.c | 11 ++++++++---
> > > > > >  1 file changed, 8 insertions(+), 3 deletions(-)
> > > > > 
> > > > > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF 
> > > > > platform
> > > > > device, and it must be probed before pcibios_init() which is a
> > > > > subsys_initcall(), or else the PCI bus never gets scanned.
> > > > 
> > > > Thanks for the report. This is probably getting dropped, but it could
> > > > be disabled for PPC.
> > > 
> > > I don't think that adding another arbitrary arch difference would be the
> > > right solution.
> > 
> > I think Rob meant temporarily disable it while things get fixed. At
> > least, 
> 
> So, what is the permanent fix for the swiotlb issue (or more generally, the 
> inability to have a late_initcall that runs after non-module, non-hotplug 
> platform devices have been probed)?
> 
> > I don't see any reason why PPC wouldn't benefit from this
> > series.
> 
> It's not clear to me what the benefit of this is at all, much less for PPC.   
> What is the fundamental problem with deferred probes?  In the cover letter 
> you say this change saves 2.3 seconds, but where is that time being consumed? 
>  Are the drivers taking too long in their probe function trying to initialize 
> and then deferring, rather than checking for dependencies up front?  Or are 
> there really so many devices and such a pessimal ordering that most of the 
> time is spent iterating through and reordering the list, with each defer 
> happening quickly?
> 
> Even if something different does need to be done at this level, forcing all 
> OF platform devices to be probed at the late_initcall level seems quite 
> intrusive.

Totally agreed.

> You limited it to OF because people complained that other things 
> will break.

Right.

And I'm not sure why that was regarded as a good enough reason to do it.

> Things still broke.

Yes, they did.

> Surely there's a better way to address the 
> problem.  Can't the delay be requested by drivers that might otherwise need 
> to defer (which could be done incrementally, focusing on the worst 
> performance problems), rather than enabling it for everything?

Well, I was suggesting to use an opt-in flag there, but I'm not sure if Tomeu
took that into consideration.

In any case, probing is just one aspect of a deeper issue, which is that
we have no way to represent functional dependencies between devices.

Thanks,
Rafael

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Tomeu Vizoso]

On 22 October 2015 at 23:27, Scott Wood <scottwood@freescale.com> wrote:
> On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
>> On 22 October 2015 at 00:51, Scott Wood <scottwood@freescale.com> wrote:
>> > On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
>> > > On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com>
>> > > wrote:
>> > > > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
>> > > > > Instead of trying to match and probe platform and AMBA devices right
>> > > > > after each is registered, delay their probes until
>> > > > > device_initcall_sync.
>> > > > >
>> > > > > This means that devices will start probing once all built-in drivers
>> > > > > have registered, and after all platform and AMBA devices from the DT
>> > > > > have been registered already.
>> > > > >
>> > > > > This allows us to prevent deferred probes by probing dependencies on
>> > > > > demand.
>> > > > >
>> > > > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>> > > > > ---
>> > > > >
>> > > > > Changes in v4:
>> > > > > - Also defer probes of AMBA devices registered from the DT as they
>> > > > > can
>> > > > >   also request resources.
>> > > > >
>> > > > >  drivers/of/platform.c | 11 ++++++++---
>> > > > >  1 file changed, 8 insertions(+), 3 deletions(-)
>> > > >
>> > > > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF
>> > > > platform
>> > > > device, and it must be probed before pcibios_init() which is a
>> > > > subsys_initcall(), or else the PCI bus never gets scanned.
>> > >
>> > > Thanks for the report. This is probably getting dropped, but it could
>> > > be disabled for PPC.
>> >
>> > I don't think that adding another arbitrary arch difference would be the
>> > right solution.
>>
>> I think Rob meant temporarily disable it while things get fixed. At
>> least,
>
> So, what is the permanent fix for the swiotlb issue (or more generally, the
> inability to have a late_initcall that runs after non-module, non-hotplug
> platform devices have been probed)?

If the code in pcibios_init() depends on the PCI bus device having
probed, then I would recommend making that dependency explicit by
calling of_device_probe() on the OF node of the PCI controller when
looking it up.

>> I don't see any reason why PPC wouldn't benefit from this
>> series.
>
> It's not clear to me what the benefit of this is at all, much less for PPC.
> What is the fundamental problem with deferred probes?  In the cover letter
> you say this change saves 2.3 seconds, but where is that time being consumed?
>  Are the drivers taking too long in their probe function trying to initialize
> and then deferring, rather than checking for dependencies up front?  Or are
> there really so many devices and such a pessimal ordering that most of the
> time is spent iterating through and reordering the list, with each defer
> happening quickly?

The problem is that a device that defers its probe is currently sent
to the back of the queue, and that's undesired in some use cases in
which there's a device that should be up as soon as possible during
boot (and boot takes a long time). So the goal is to change the order
in which devices with dependencies end up probing.

> Even if something different does need to be done at this level, forcing all
> OF platform devices to be probed at the late_initcall level seems quite
> intrusive.  You limited it to OF because people complained that other things
> will break.  Things still broke.  Surely there's a better way to address the
> problem.  Can't the delay be requested by drivers that might otherwise need
> to defer (which could be done incrementally, focusing on the worst
> performance problems), rather than enabling it for everything?

Yes, given the amount of breakage that's a sensible option. But in any
case and even if this series is most likely to be dropped, I recommend
to make explicit as many implicit dependencies as possible.

Regards,

Tomeu

> -Scott
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Rob Herring]

On Wed, Oct 28, 2015 at 9:40 AM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
> On 22 October 2015 at 23:27, Scott Wood <scottwood@freescale.com> wrote:
>> On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
>>> On 22 October 2015 at 00:51, Scott Wood <scottwood@freescale.com> wrote:
>>> > On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
>>> > > On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <scottwood@freescale.com>
>>> > > wrote:
>>> > > > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
>>> > > > > Instead of trying to match and probe platform and AMBA devices right
>>> > > > > after each is registered, delay their probes until
>>> > > > > device_initcall_sync.
>>> > > > >
>>> > > > > This means that devices will start probing once all built-in drivers
>>> > > > > have registered, and after all platform and AMBA devices from the DT
>>> > > > > have been registered already.
>>> > > > >
>>> > > > > This allows us to prevent deferred probes by probing dependencies on
>>> > > > > demand.
>>> > > > >
>>> > > > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>>> > > > > ---
>>> > > > >
>>> > > > > Changes in v4:
>>> > > > > - Also defer probes of AMBA devices registered from the DT as they
>>> > > > > can
>>> > > > >   also request resources.
>>> > > > >
>>> > > > >  drivers/of/platform.c | 11 ++++++++---
>>> > > > >  1 file changed, 8 insertions(+), 3 deletions(-)
>>> > > >
>>> > > > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF
>>> > > > platform
>>> > > > device, and it must be probed before pcibios_init() which is a
>>> > > > subsys_initcall(), or else the PCI bus never gets scanned.
>>> > >
>>> > > Thanks for the report. This is probably getting dropped, but it could
>>> > > be disabled for PPC.
>>> >
>>> > I don't think that adding another arbitrary arch difference would be the
>>> > right solution.
>>>
>>> I think Rob meant temporarily disable it while things get fixed. At
>>> least,
>>
>> So, what is the permanent fix for the swiotlb issue (or more generally, the
>> inability to have a late_initcall that runs after non-module, non-hotplug
>> platform devices have been probed)?
>
> If the code in pcibios_init() depends on the PCI bus device having
> probed, then I would recommend making that dependency explicit by
> calling of_device_probe() on the OF node of the PCI controller when
> looking it up.
>
>>> I don't see any reason why PPC wouldn't benefit from this
>>> series.
>>
>> It's not clear to me what the benefit of this is at all, much less for PPC.
>> What is the fundamental problem with deferred probes?  In the cover letter
>> you say this change saves 2.3 seconds, but where is that time being consumed?
>>  Are the drivers taking too long in their probe function trying to initialize
>> and then deferring, rather than checking for dependencies up front?  Or are
>> there really so many devices and such a pessimal ordering that most of the
>> time is spent iterating through and reordering the list, with each defer
>> happening quickly?
>
> The problem is that a device that defers its probe is currently sent
> to the back of the queue, and that's undesired in some use cases in
> which there's a device that should be up as soon as possible during
> boot (and boot takes a long time). So the goal is to change the order
> in which devices with dependencies end up probing.
>
>> Even if something different does need to be done at this level, forcing all
>> OF platform devices to be probed at the late_initcall level seems quite
>> intrusive.  You limited it to OF because people complained that other things
>> will break.  Things still broke.  Surely there's a better way to address the
>> problem.  Can't the delay be requested by drivers that might otherwise need
>> to defer (which could be done incrementally, focusing on the worst
>> performance problems), rather than enabling it for everything?
>
> Yes, given the amount of breakage that's a sensible option. But in any
> case and even if this series is most likely to be dropped, I recommend
> to make explicit as many implicit dependencies as possible.

It is dropped for now, but I expect much of this to still integrate
with Rafael's proposal. However, it will need to be opt-in it seems.
That is somewhat unfortunate because things like async probe are
opt-in and seem to rarely get used. As to how we can make it opt-in, I
think we can make of_platform_populate() callable multiple times at
the root level. This will allow a platform to skip calling it and then
a late call to it can register the devices after drivers have
registered. This late call will then be a nop for platforms that don't
opt-in and call of_platform_populate themselves.

Rob

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Scott Wood]

On Wed, 2015-10-28 at 15:40 +0100, Tomeu Vizoso wrote:
> On 22 October 2015 at 23:27, Scott Wood <scottwood@freescale.com> wrote:
> > On Thu, 2015-10-22 at 15:04 +0200, Tomeu Vizoso wrote:
> > > On 22 October 2015 at 00:51, Scott Wood <scottwood@freescale.com> wrote:
> > > > On Wed, 2015-10-21 at 08:44 -0500, Rob Herring wrote:
> > > > > On Wed, Oct 21, 2015 at 12:54 AM, Scott Wood <
> > > > > scottwood@freescale.com>
> > > > > wrote:
> > > > > > On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:
> > > > > > > Instead of trying to match and probe platform and AMBA devices 
> > > > > > > right
> > > > > > > after each is registered, delay their probes until
> > > > > > > device_initcall_sync.
> > > > > > > 
> > > > > > > This means that devices will start probing once all built-in 
> > > > > > > drivers
> > > > > > > have registered, and after all platform and AMBA devices from 
> > > > > > > the DT
> > > > > > > have been registered already.
> > > > > > > 
> > > > > > > This allows us to prevent deferred probes by probing 
> > > > > > > dependencies on
> > > > > > > demand.
> > > > > > > 
> > > > > > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > > > > > > ---
> > > > > > > 
> > > > > > > Changes in v4:
> > > > > > > - Also defer probes of AMBA devices registered from the DT as 
> > > > > > > they
> > > > > > > can
> > > > > > >   also request resources.
> > > > > > > 
> > > > > > >  drivers/of/platform.c | 11 ++++++++---
> > > > > > >  1 file changed, 8 insertions(+), 3 deletions(-)
> > > > > > 
> > > > > > This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF
> > > > > > platform
> > > > > > device, and it must be probed before pcibios_init() which is a
> > > > > > subsys_initcall(), or else the PCI bus never gets scanned.
> > > > > 
> > > > > Thanks for the report. This is probably getting dropped, but it 
> > > > > could
> > > > > be disabled for PPC.
> > > > 
> > > > I don't think that adding another arbitrary arch difference would be 
> > > > the
> > > > right solution.
> > > 
> > > I think Rob meant temporarily disable it while things get fixed. At
> > > least,
> > 
> > So, what is the permanent fix for the swiotlb issue (or more generally, 
> > the
> > inability to have a late_initcall that runs after non-module, non-hotplug
> > platform devices have been probed)?
> 
> If the code in pcibios_init() depends on the PCI bus device having
> probed, then I would recommend making that dependency explicit by
> calling of_device_probe() on the OF node of the PCI controller when
> looking it up.

"when looking it up"?  pcibios_init() doesn't do anything with the OF node or 
know any details about the particular PCI bus host implementation.

> > > I don't see any reason why PPC wouldn't benefit from this
> > > series.
> > 
> > It's not clear to me what the benefit of this is at all, much less for 
> > PPC.
> > What is the fundamental problem with deferred probes?  In the cover letter
> > you say this change saves 2.3 seconds, but where is that time being 
> > consumed?
> >  Are the drivers taking too long in their probe function trying to 
> > initialize
> > and then deferring, rather than checking for dependencies up front?  Or 
> > are
> > there really so many devices and such a pessimal ordering that most of the
> > time is spent iterating through and reordering the list, with each defer
> > happening quickly?
> 
> The problem is that a device that defers its probe is currently sent
> to the back of the queue, and that's undesired in some use cases in
> which there's a device that should be up as soon as possible during
> boot (and boot takes a long time). So the goal is to change the order
> in which devices with dependencies end up probing.

That doesn't answer my question about where the time is being spent.  Did you 
profile?

-Scott

Re: [PATCH v6 22/22] of/platform: Defer probes of registered devices

[Michael Ellerman]

On Wed, 2015-10-21 at 00:54 -0500, Scott Wood wrote:

> On Mon, 2015-09-21 at 16:03 +0200, Tomeu Vizoso wrote:

> > Instead of trying to match and probe platform and AMBA devices right
> > after each is registered, delay their probes until device_initcall_sync.
> > 
> > This means that devices will start probing once all built-in drivers
> > have registered, and after all platform and AMBA devices from the DT
> > have been registered already.
> > 
> > This allows us to prevent deferred probes by probing dependencies on
> > demand.
> > 
> > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > ---
> > 
> > Changes in v4:
> > - Also defer probes of AMBA devices registered from the DT as they can
> >   also request resources.
> > 
> >  drivers/of/platform.c | 11 ++++++++---
> >  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> This breaks arch/powerpc/sysdev/fsl_pci.c.  The PCI bus is an OF platform 
> device, and it must be probed before pcibios_init() which is a 
> subsys_initcall(), or else the PCI bus never gets scanned.

Ah right. This is presumably why I'm not seeing any PCI devices on my p5020ds
with linux-next.

cheers