[PATCH 00/21] On-demand device registration

[PATCH 00/21] On-demand device registration

[Tomeu Vizoso]

Hello,

I have a problem with the panel on my Tegra Chromebook taking longer than
expected to be ready during boot (Stéphane Marchesin reported what is
basically the same issue in [0]), and have looked into ordered probing as a
better way of solving this than moving nodes around in the DT or playing with
initcall levels.

While reading the thread [1] that Alexander Holler started with his series to
make probing order deterministic, it occurred to me that it should be possible
to achieve the same by registering devices as they are referenced by other
devices.

This basically reuses the information that is already implicit in the probe()
implementations, saving us from refactoring existing drivers or adding
information to DTBs.

Something I'm not completely happy with is that I have had to move the call to
of_platform_populate after all platform drivers have been registered.
Otherwise I don't see how I could register drivers on demand as we don't have
yet each driver's compatible strings.

For machs that don't move of_platform_populate() to a later point, these
patches shouldn't cause any problems but it's not guaranteed that we'll avoid
all the deferred probes as some drivers may not be registered yet.

I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
patches were enough to eliminate all the deferred probes.

With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.

Regards,

Tomeu

[0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html

[1] https://lkml.org/lkml/2014/5/12/452

Tomeu Vizoso (21):
  regulator: core: Reduce critical area in _regulator_get
  ARM: tegra: Add gpio-ranges property
  ARM: tegra: Register drivers before devices
  ARM: EXYNOS: Register drivers before devices
  ARM i.MX6q: Register drivers before devices
  of/platform: Add of_platform_device_ensure()
  of/platform: Ensure device registration on lookup
  gpio: Probe GPIO drivers on demand
  gpio: Probe pinctrl devices on demand
  regulator: core: Probe regulators on demand
  drm: Probe panels on demand
  drm/tegra: Probe dpaux devices on demand
  i2c: core: Probe i2c master devices on demand
  pwm: Probe PWM chip devices on demand
  backlight: Probe backlight devices on demand
  usb: phy: Probe phy devices on demand
  clk: Probe clk providers on demand
  pinctrl: Probe pinctrl devices on demand
  phy: core: Probe phy providers on demand
  dma: of: Probe DMA controllers on demand
  power-supply: Probe power supplies on demand

 arch/arm/boot/dts/tegra124.dtsi     |  1 +
 arch/arm/mach-exynos/exynos.c       |  4 +--
 arch/arm/mach-imx/mach-imx6q.c      | 12 ++++-----
 arch/arm/mach-tegra/tegra.c         | 21 ++++++---------
 drivers/clk/clk.c                   |  3 +++
 drivers/dma/of-dma.c                |  3 +++
 drivers/gpio/gpiolib-of.c           |  5 ++++
 drivers/gpu/drm/drm_panel.c         |  3 +++
 drivers/gpu/drm/tegra/dpaux.c       |  3 +++
 drivers/i2c/i2c-core.c              |  3 +++
 drivers/of/platform.c               | 53 +++++++++++++++++++++++++++++++++++++
 drivers/phy/phy-core.c              |  3 +++
 drivers/pinctrl/devicetree.c        |  2 ++
 drivers/power/power_supply_core.c   |  3 +++
 drivers/pwm/core.c                  |  3 +++
 drivers/regulator/core.c            | 45 +++++++++++++++----------------
 drivers/usb/phy/phy.c               |  3 +++
 drivers/video/backlight/backlight.c |  3 +++
 include/linux/of_platform.h         |  2 ++
 19 files changed, 130 insertions(+), 45 deletions(-)

-- 
2.4.1

[PATCH 17/21] clk: Probe clk providers on demand

[Tomeu Vizoso]

When looking up a clk through its DT node, ensure that the corresponding
device has been registered.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
---
 drivers/clk/clk.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index 5b0f418..f9e8a8d 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -18,6 +18,7 @@
 #include <linux/list.h>
 #include <linux/slab.h>
 #include <linux/of.h>
+#include <linux/of_platform.h>
 #include <linux/device.h>
 #include <linux/init.h>
 #include <linux/sched.h>
@@ -2985,6 +2986,8 @@ struct clk *__of_clk_get_from_provider(struct of_phandle_args *clkspec,
 	if (!clkspec)
 		return ERR_PTR(-EINVAL);
 
+	of_platform_device_ensure(clkspec->np);
+
 	/* Check if we have such a provider in our array */
 	mutex_lock(&of_clk_mutex);
 	list_for_each_entry(provider, &of_clk_providers, link) {
-- 
2.4.1

Re: [PATCH 17/21] clk: Probe clk providers on demand

[Michael Turquette]

Quoting Tomeu Vizoso (2015-05-25 07:53:21)
> When looking up a clk through its DT node, ensure that the corresponding
> device has been registered.
> =

> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>

Ack.

Regards,
Mike

> ---
>  drivers/clk/clk.c | 3 +++
>  1 file changed, 3 insertions(+)
> =

> diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
> index 5b0f418..f9e8a8d 100644
> --- a/drivers/clk/clk.c
> +++ b/drivers/clk/clk.c
> @@ -18,6 +18,7 @@
>  #include <linux/list.h>
>  #include <linux/slab.h>
>  #include <linux/of.h>
> +#include <linux/of_platform.h>
>  #include <linux/device.h>
>  #include <linux/init.h>
>  #include <linux/sched.h>
> @@ -2985,6 +2986,8 @@ struct clk *__of_clk_get_from_provider(struct of_ph=
andle_args *clkspec,
>         if (!clkspec)
>                 return ERR_PTR(-EINVAL);
>  =

> +       of_platform_device_ensure(clkspec->np);
> +
>         /* Check if we have such a provider in our array */
>         mutex_lock(&of_clk_mutex);
>         list_for_each_entry(provider, &of_clk_providers, link) {
> -- =

> 2.4.1
>=20

Re: [PATCH 00/21] On-demand device registration

[Rob Herring]

On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
> Hello,
>
> I have a problem with the panel on my Tegra Chromebook taking longer than
> expected to be ready during boot (Stéphane Marchesin reported what is
> basically the same issue in [0]), and have looked into ordered probing as a
> better way of solving this than moving nodes around in the DT or playing with
> initcall levels.
>
> While reading the thread [1] that Alexander Holler started with his series to
> make probing order deterministic, it occurred to me that it should be possible
> to achieve the same by registering devices as they are referenced by other
> devices.

I like the concept and novel approach.

> This basically reuses the information that is already implicit in the probe()
> implementations, saving us from refactoring existing drivers or adding
> information to DTBs.
>
> Something I'm not completely happy with is that I have had to move the call to
> of_platform_populate after all platform drivers have been registered.
> Otherwise I don't see how I could register drivers on demand as we don't have
> yet each driver's compatible strings.

Yeah, this is the opposite of what we'd really like. Ideally, we would
have a solution that works for modules too. However, we're no worse
off. We pretty much build-in dependencies to avoid module ordering
problems.

Perhaps we need to make the probing on-demand rather than simply on
device<->driver match occurring.

> For machs that don't move of_platform_populate() to a later point, these
> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
> all the deferred probes as some drivers may not be registered yet.

Ideally, of_platform_populate is not explicitly called by each
platform. So I think we need to make this work for the default case.

> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
> patches were enough to eliminate all the deferred probes.
>
> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.

That's certainly compelling.

Rob

>
> Regards,
>
> Tomeu
>
> [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html
>
> [1] https://lkml.org/lkml/2014/5/12/452
>
> Tomeu Vizoso (21):
>   regulator: core: Reduce critical area in _regulator_get
>   ARM: tegra: Add gpio-ranges property
>   ARM: tegra: Register drivers before devices
>   ARM: EXYNOS: Register drivers before devices
>   ARM i.MX6q: Register drivers before devices
>   of/platform: Add of_platform_device_ensure()
>   of/platform: Ensure device registration on lookup
>   gpio: Probe GPIO drivers on demand
>   gpio: Probe pinctrl devices on demand
>   regulator: core: Probe regulators on demand
>   drm: Probe panels on demand
>   drm/tegra: Probe dpaux devices on demand
>   i2c: core: Probe i2c master devices on demand
>   pwm: Probe PWM chip devices on demand
>   backlight: Probe backlight devices on demand
>   usb: phy: Probe phy devices on demand
>   clk: Probe clk providers on demand
>   pinctrl: Probe pinctrl devices on demand
>   phy: core: Probe phy providers on demand
>   dma: of: Probe DMA controllers on demand
>   power-supply: Probe power supplies on demand
>
>  arch/arm/boot/dts/tegra124.dtsi     |  1 +
>  arch/arm/mach-exynos/exynos.c       |  4 +--
>  arch/arm/mach-imx/mach-imx6q.c      | 12 ++++-----
>  arch/arm/mach-tegra/tegra.c         | 21 ++++++---------
>  drivers/clk/clk.c                   |  3 +++
>  drivers/dma/of-dma.c                |  3 +++
>  drivers/gpio/gpiolib-of.c           |  5 ++++
>  drivers/gpu/drm/drm_panel.c         |  3 +++
>  drivers/gpu/drm/tegra/dpaux.c       |  3 +++
>  drivers/i2c/i2c-core.c              |  3 +++
>  drivers/of/platform.c               | 53 +++++++++++++++++++++++++++++++++++++
>  drivers/phy/phy-core.c              |  3 +++
>  drivers/pinctrl/devicetree.c        |  2 ++
>  drivers/power/power_supply_core.c   |  3 +++
>  drivers/pwm/core.c                  |  3 +++
>  drivers/regulator/core.c            | 45 +++++++++++++++----------------
>  drivers/usb/phy/phy.c               |  3 +++
>  drivers/video/backlight/backlight.c |  3 +++
>  include/linux/of_platform.h         |  2 ++
>  19 files changed, 130 insertions(+), 45 deletions(-)
>
> --
> 2.4.1
>

Re: [PATCH 00/21] On-demand device registration

[Grygorii.Strashko@linaro.org]

Hi Tomeu,

On 05/28/2015 07:33 AM, Rob Herring wrote:
> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso <tomeu.vizoso@collabora.com> wrote:
>> I have a problem with the panel on my Tegra Chromebook taking longer than
>> expected to be ready during boot (Stéphane Marchesin reported what is
>> basically the same issue in [0]), and have looked into ordered probing as a
>> better way of solving this than moving nodes around in the DT or playing with
>> initcall levels.
>>
>> While reading the thread [1] that Alexander Holler started with his series to
>> make probing order deterministic, it occurred to me that it should be possible
>> to achieve the same by registering devices as they are referenced by other
>> devices.
> 
> I like the concept and novel approach.
> 
>> This basically reuses the information that is already implicit in the probe()
>> implementations, saving us from refactoring existing drivers or adding
>> information to DTBs.
>>
>> Something I'm not completely happy with is that I have had to move the call to
>> of_platform_populate after all platform drivers have been registered.
>> Otherwise I don't see how I could register drivers on demand as we don't have
>> yet each driver's compatible strings.
> 
> Yeah, this is the opposite of what we'd really like. Ideally, we would
> have a solution that works for modules too. However, we're no worse
> off. We pretty much build-in dependencies to avoid module ordering
> problems.
> 
> Perhaps we need to make the probing on-demand rather than simply on
> device<->driver match occurring.
> 
>> For machs that don't move of_platform_populate() to a later point, these
>> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
>> all the deferred probes as some drivers may not be registered yet.
> 
> Ideally, of_platform_populate is not explicitly called by each
> platform. So I think we need to make this work for the default case.
> 
>> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
>> patches were enough to eliminate all the deferred probes.
>>
>> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.
> 
> That's certainly compelling.

I've found your idea about moving device registration later during System boot
very interesting so I've decided to try it on dra7-evem (TI) :).
It's good to know time during Kernel boot when we can assume that all drivers are
ready for probing, so there are more ways to control probing order.

Pls, Note here that TI OMAP2+ mach is not pure DT mach - it's combination of
DT and not DT devices/drivers.

Ok. So What was done...

LKML Linux 4.1-rc3 (a simple case)
1) use your patches 3/4 as reference (only these two patches :)
2) move of_platform_populate later at device_initcall_sync time
Boot time reduction ~0.4 sec


TI Android Kernel 3.14 (NOT a simple case)
1) use your patches 3/4 as reference (only these two patches :)
2) move of_platform_populate later at device_initcall_sync time
3) make it to boot (not sure I've fixed all issues, but those which
   break the System boot):
 - split non-DT and DT devices registration in platform code;
 - keep non-DT devices registration from .init_machine() [arch_initcall]
 - move DT-devices registration at device_initcall_sync time
 - fix drivers which use platform_driver_probe().
   Note. Now there are at about ~190 occurrences of this macro in Kernel.
 - re-order few devices in DT (4 devices)
 - fix one driver which uses of_find_device_by_node() wrongly
   Note. This API is used some times with assumption that
   requested dev has been probed already.
Boot time reduction ~0.3 sec. Probing of some devices are still deferred.

TI Android Kernel 3.14 + of_platform_device_ensure
1) backport of_platform_device_ensure() (also need patches 
   "of: Introduce device tree node flag helpers" and 
   "of: Keep track of populated platform devices")
2) back-port all your patches which uses of_platform_device_ensure()
3) make it to boot:
   - drop patch dma: of: Probe DMA controllers on demand
   - fix deadlock in regulator core:
   regulator_dev_lookup() called from regulator_register() in K3.14
4) get rid of deferred probes - add of_platform_device_ensure() calls in:
   - drivers/video/fbdev/omap2/dss/output.c
   - drivers/extcon/extcon-class.c

 Boot time reduction: NONE !?


So few comments from above:
- registering devices later during the System boot may improve boot time.
  But resolving of all deferred probes may NOT improve boot time ;) 
  Have you seen smth like this?

- usage of of_platform_device_ensure() will require continuous fixing of Kernel :(

- late_initcall is not (as for me not safe) a good time to register devices. A lot
  of platforms/subsystems/frameworks perform their final initialization or clean-up
  steps, with assumption that System mostly ready to work. For example, CPUIdle/CPUFreq
  are allowed and other PM staff. CPUIdle and driver's probing are not friends.

What would be nice to have for now in my opinion:
- some useful place to move device population code.
  May be machine_desc->init_device_sync() callback.
- some optional feature in DTC which will allow to re-arrange DT nodes for levels 0(root)
  and 1(simple-bus) using standard or widely used bindings (gpio, regulators, clocks, dma,
  pinctrl, irqs).

Additional note:
- Changing device's registration order will change devices's order in
 dpm_list (drivers/base/power/main.c) and devices_kset list
 (drivers/base/core.c). This might potentially affect on suspend [1] and shutdown. 
[1] https://lkml.org/lkml/2014/12/12/324


-- 
regards,
-grygorii

Re: [PATCH 00/21] On-demand device registration

[Tomeu Vizoso]

On 3 June 2015 at 21:57, Grygorii.Strashko@linaro.org
<grygorii.strashko@linaro.org> wrote:
> Hi Tomeu,
>
> On 05/28/2015 07:33 AM, Rob Herring wrote:
>> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso <tomeu.vizoso@collabora.com> wrote:
>>> I have a problem with the panel on my Tegra Chromebook taking longer than
>>> expected to be ready during boot (Stéphane Marchesin reported what is
>>> basically the same issue in [0]), and have looked into ordered probing as a
>>> better way of solving this than moving nodes around in the DT or playing with
>>> initcall levels.
>>>
>>> While reading the thread [1] that Alexander Holler started with his series to
>>> make probing order deterministic, it occurred to me that it should be possible
>>> to achieve the same by registering devices as they are referenced by other
>>> devices.
>>
>> I like the concept and novel approach.
>>
>>> This basically reuses the information that is already implicit in the probe()
>>> implementations, saving us from refactoring existing drivers or adding
>>> information to DTBs.
>>>
>>> Something I'm not completely happy with is that I have had to move the call to
>>> of_platform_populate after all platform drivers have been registered.
>>> Otherwise I don't see how I could register drivers on demand as we don't have
>>> yet each driver's compatible strings.
>>
>> Yeah, this is the opposite of what we'd really like. Ideally, we would
>> have a solution that works for modules too. However, we're no worse
>> off. We pretty much build-in dependencies to avoid module ordering
>> problems.
>>
>> Perhaps we need to make the probing on-demand rather than simply on
>> device<->driver match occurring.
>>
>>> For machs that don't move of_platform_populate() to a later point, these
>>> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
>>> all the deferred probes as some drivers may not be registered yet.
>>
>> Ideally, of_platform_populate is not explicitly called by each
>> platform. So I think we need to make this work for the default case.
>>
>>> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
>>> patches were enough to eliminate all the deferred probes.
>>>
>>> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.
>>
>> That's certainly compelling.
>
> I've found your idea about moving device registration later during System boot
> very interesting so I've decided to try it on dra7-evem (TI) :).
> It's good to know time during Kernel boot when we can assume that all drivers are
> ready for probing, so there are more ways to control probing order.

Thanks, though right now I'm following Rob's suggestion and only delay
probing, not registration. The patch is really simple (applies on
linux-next, with async probing):

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 8da8e07..7e6b1e1 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -407,6 +407,11 @@ int driver_probe_device(struct device_driver
*drv, struct device *dev)
        if (!device_is_registered(dev))
                return -ENODEV;

+       if (!driver_deferred_probe_enable) {
+               driver_deferred_probe_add(dev);
+               return 0;
+       }
+
        pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
                 drv->bus->name, __func__, dev_name(dev), drv->name);

@@ -585,7 +590,7 @@ EXPORT_SYMBOL_GPL(device_attach);

 void device_initial_probe(struct device *dev)
 {
-       __device_attach(dev, true);
+       __device_attach(dev, driver_deferred_probe_enable);
 }

 static int __driver_attach(struct device *dev, void *data)

> Pls, Note here that TI OMAP2+ mach is not pure DT mach - it's combination of
> DT and not DT devices/drivers.
>
> Ok. So What was done...
>
> LKML Linux 4.1-rc3 (a simple case)
> 1) use your patches 3/4 as reference (only these two patches :)
> 2) move of_platform_populate later at device_initcall_sync time
> Boot time reduction ~0.4 sec

I'm a bit surprised at such a big improvement. May I ask how you are
measuring it?

> TI Android Kernel 3.14 (NOT a simple case)
> 1) use your patches 3/4 as reference (only these two patches :)
> 2) move of_platform_populate later at device_initcall_sync time
> 3) make it to boot (not sure I've fixed all issues, but those which
>    break the System boot):
>  - split non-DT and DT devices registration in platform code;
>  - keep non-DT devices registration from .init_machine() [arch_initcall]
>  - move DT-devices registration at device_initcall_sync time
>  - fix drivers which use platform_driver_probe().
>    Note. Now there are at about ~190 occurrences of this macro in Kernel.
>  - re-order few devices in DT (4 devices)
>  - fix one driver which uses of_find_device_by_node() wrongly
>    Note. This API is used some times with assumption that
>    requested dev has been probed already.
> Boot time reduction ~0.3 sec. Probing of some devices are still deferred.

I got no deferred probes on a pandaboard with just these changes:

https://git.collabora.com/cgit/user/tomeu/linux.git/commit/?id=1586c6f50b3d3c65dc219a3cdc3327d798cabca6

> TI Android Kernel 3.14 + of_platform_device_ensure
> 1) backport of_platform_device_ensure() (also need patches
>    "of: Introduce device tree node flag helpers" and
>    "of: Keep track of populated platform devices")
> 2) back-port all your patches which uses of_platform_device_ensure()
> 3) make it to boot:
>    - drop patch dma: of: Probe DMA controllers on demand
>    - fix deadlock in regulator core:
>    regulator_dev_lookup() called from regulator_register() in K3.14
> 4) get rid of deferred probes - add of_platform_device_ensure() calls in:
>    - drivers/video/fbdev/omap2/dss/output.c
>    - drivers/extcon/extcon-class.c
>
>  Boot time reduction: NONE !?
>
> So few comments from above:
> - registering devices later during the System boot may improve boot time.
>   But resolving of all deferred probes may NOT improve boot time ;)
>   Have you seen smth like this?

Yeah, I don't really expect for on-demand probing to improve boot time
much in the general case, as drivers tend to first acquire resources
(and defer probe if needed) and only then access hardware and wait for
stuff to happen.

The main advantage of ordered/ondemand probing is that it is much
easier to see the order in which devices will be bound. In my case
this is useful because one could get one device (eg. the drm panel) to
probe very early by just moving that node to the beginning of the DT.
With deferred probe, one would have to figure out all the dependencies
and shuffle them around in the DT.

Another advantage (but not the one why I'm doing this work) is that in
general a driver's probe will be called only once per device, and if
it fails then we can be almost certain that something is wrong. This
should aid in debugging as right now one has to take into account the
several reasons why a device might defer its probe.

Depending on what work your drivers do in your platform, enabling
async probing for all of them may have a noticeable impact though.

> - usage of of_platform_device_ensure() will require continuous fixing of Kernel :(

You mean adding those calls to more subsystems?

> - late_initcall is not (as for me not safe) a good time to register devices. A lot
>   of platforms/subsystems/frameworks perform their final initialization or clean-up
>   steps, with assumption that System mostly ready to work. For example, CPUIdle/CPUFreq
>   are allowed and other PM staff. CPUIdle and driver's probing are not friends.

Yeah, I was expecting to find more problems due to this, but the
platforms I tested on didn't show any. Do you have pointers to
concrete issues?

> What would be nice to have for now in my opinion:
> - some useful place to move device population code.
>   May be machine_desc->init_device_sync() callback.

I'm looking at leave device registration where it currently is and
just add devices to the deferred list until we get to late_initcall,
where we would start to actually probe them. Seems promising for now.

> - some optional feature in DTC which will allow to re-arrange DT nodes for levels 0(root)
>   and 1(simple-bus) using standard or widely used bindings (gpio, regulators, clocks, dma,
>   pinctrl, irqs).

Could you extend on this, please?

> Additional note:
> - Changing device's registration order will change devices's order in
>  dpm_list (drivers/base/power/main.c) and devices_kset list
>  (drivers/base/core.c). This might potentially affect on suspend [1] and shutdown.

Yeah, I don't plan to change registration order, but probing certainly will.

> [1] https://lkml.org/lkml/2014/12/12/324

Thanks,

Tomeu

>
> --
> regards,
> -grygorii
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 00/21] On-demand device registration

[Grygorii.Strashko@linaro.org]

On 06/04/2015 11:39 AM, Tomeu Vizoso wrote:
> On 3 June 2015 at 21:57, Grygorii.Strashko@linaro.org
> <grygorii.strashko@linaro.org> wrote:
>> On 05/28/2015 07:33 AM, Rob Herring wrote:
>>> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso <tomeu.vizoso@collabora.com> wrote:
>>>> I have a problem with the panel on my Tegra Chromebook taking longer than
>>>> expected to be ready during boot (Stéphane Marchesin reported what is
>>>> basically the same issue in [0]), and have looked into ordered probing as a
>>>> better way of solving this than moving nodes around in the DT or playing with
>>>> initcall levels.
>>>>
>>>> While reading the thread [1] that Alexander Holler started with his series to
>>>> make probing order deterministic, it occurred to me that it should be possible
>>>> to achieve the same by registering devices as they are referenced by other
>>>> devices.
>>>
>>> I like the concept and novel approach.
>>>
>>>> This basically reuses the information that is already implicit in the probe()
>>>> implementations, saving us from refactoring existing drivers or adding
>>>> information to DTBs.
>>>>
>>>> Something I'm not completely happy with is that I have had to move the call to
>>>> of_platform_populate after all platform drivers have been registered.
>>>> Otherwise I don't see how I could register drivers on demand as we don't have
>>>> yet each driver's compatible strings.
>>>
>>> Yeah, this is the opposite of what we'd really like. Ideally, we would
>>> have a solution that works for modules too. However, we're no worse
>>> off. We pretty much build-in dependencies to avoid module ordering
>>> problems.
>>>
>>> Perhaps we need to make the probing on-demand rather than simply on
>>> device<->driver match occurring.
>>>
>>>> For machs that don't move of_platform_populate() to a later point, these
>>>> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
>>>> all the deferred probes as some drivers may not be registered yet.
>>>
>>> Ideally, of_platform_populate is not explicitly called by each
>>> platform. So I think we need to make this work for the default case.
>>>
>>>> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
>>>> patches were enough to eliminate all the deferred probes.
>>>>
>>>> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.
>>>
>>> That's certainly compelling.
>>
>> I've found your idea about moving device registration later during System boot
>> very interesting so I've decided to try it on dra7-evem (TI) :).
>> It's good to know time during Kernel boot when we can assume that all drivers are
>> ready for probing, so there are more ways to control probing order.
> 
> Thanks, though right now I'm following Rob's suggestion and only delay
> probing, not registration. The patch is really simple (applies on
> linux-next, with async probing):
> 
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index 8da8e07..7e6b1e1 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -407,6 +407,11 @@ int driver_probe_device(struct device_driver
> *drv, struct device *dev)
>          if (!device_is_registered(dev))
>                  return -ENODEV;
> 
> +       if (!driver_deferred_probe_enable) {
> +               driver_deferred_probe_add(dev);
> +               return 0;
> +       }
> +
>          pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
>                   drv->bus->name, __func__, dev_name(dev), drv->name);
> 
> @@ -585,7 +590,7 @@ EXPORT_SYMBOL_GPL(device_attach);
> 
>   void device_initial_probe(struct device *dev)
>   {
> -       __device_attach(dev, true);
> +       __device_attach(dev, driver_deferred_probe_enable);
>   }
> 
>   static int __driver_attach(struct device *dev, void *data)

Can't boot my 3.14 kernel with this change :( Most probably,
the problem is related to platform_driver_probe() usage :(
Have no time to play with it now :(, but recommend you to check also
earlyprintk, last log message I can see:
[    1.435522] bootconsole [earlycon0] disabled

But, nice try ;) Seems -EPROBE_DEFER is reality of life which
has to be accepted as is.

> 
>> Pls, Note here that TI OMAP2+ mach is not pure DT mach - it's combination of
>> DT and not DT devices/drivers.
>>
>> Ok. So What was done...
>>
>> LKML Linux 4.1-rc3 (a simple case)
>> 1) use your patches 3/4 as reference (only these two patches :)
>> 2) move of_platform_populate later at device_initcall_sync time
>> Boot time reduction ~0.4 sec
> 
> I'm a bit surprised at such a big improvement. May I ask how you are
> measuring it?

Ah. My measurements are not precise. I've just tracking time of message
"[    4.110756] Freeing unused kernel memory: 344K (c0994000 - c09ea000)"

> 
>> TI Android Kernel 3.14 (NOT a simple case)
>> 1) use your patches 3/4 as reference (only these two patches :)
>> 2) move of_platform_populate later at device_initcall_sync time
>> 3) make it to boot (not sure I've fixed all issues, but those which
>>     break the System boot):
>>   - split non-DT and DT devices registration in platform code;
>>   - keep non-DT devices registration from .init_machine() [arch_initcall]
>>   - move DT-devices registration at device_initcall_sync time
>>   - fix drivers which use platform_driver_probe().
>>     Note. Now there are at about ~190 occurrences of this macro in Kernel.
>>   - re-order few devices in DT (4 devices)
>>   - fix one driver which uses of_find_device_by_node() wrongly
>>     Note. This API is used some times with assumption that
>>     requested dev has been probed already.
>> Boot time reduction ~0.3 sec. Probing of some devices are still deferred.
> 
> I got no deferred probes on a pandaboard with just these changes:
> 
> https://git.collabora.com/cgit/user/tomeu/linux.git/commit/?id=1586c6f50b3d3c65dc219a3cdc3327d798cabca6

As I've mentioned I tried TI Android Kernel 3.14 where all DRA7 SoC features are enabled.
It is very close to production SW. LKML, by default, enables
mostly nothing and not all features are supported. 
In my prev exercise I was able to boot till Android GUI and it has worked :)

Also, cool statistic - really_probe() was called 136 times for me
(successful execution of dev->bus->probe() and drv->probe()).

> 
>> TI Android Kernel 3.14 + of_platform_device_ensure
>> 1) backport of_platform_device_ensure() (also need patches
>>     "of: Introduce device tree node flag helpers" and
>>     "of: Keep track of populated platform devices")
>> 2) back-port all your patches which uses of_platform_device_ensure()
>> 3) make it to boot:
>>     - drop patch dma: of: Probe DMA controllers on demand
>>     - fix deadlock in regulator core:
>>     regulator_dev_lookup() called from regulator_register() in K3.14
>> 4) get rid of deferred probes - add of_platform_device_ensure() calls in:
>>     - drivers/video/fbdev/omap2/dss/output.c
>>     - drivers/extcon/extcon-class.c
>>
>>   Boot time reduction: NONE !?
>>
>> So few comments from above:
>> - registering devices later during the System boot may improve boot time.
>>    But resolving of all deferred probes may NOT improve boot time ;)
>>    Have you seen smth like this?
> 
> Yeah, I don't really expect for on-demand probing to improve boot time
> much in the general case, as drivers tend to first acquire resources
> (and defer probe if needed) and only then access hardware and wait for
> stuff to happen.
> 
> The main advantage of ordered/ondemand probing is that it is much
> easier to see the order in which devices will be bound. In my case
> this is useful because one could get one device (eg. the drm panel) to
> probe very early by just moving that node to the beginning of the DT.
> With deferred probe, one would have to figure out all the dependencies
> and shuffle them around in the DT.
> 
> Another advantage (but not the one why I'm doing this work) is that in
> general a driver's probe will be called only once per device, and if
> it fails then we can be almost certain that something is wrong. This
> should aid in debugging as right now one has to take into account the
> several reasons why a device might defer its probe.
> 
> Depending on what work your drivers do in your platform, enabling
> async probing for all of them may have a noticeable impact though.
> 
>> - usage of of_platform_device_ensure() will require continuous fixing of Kernel :(
> 
> You mean adding those calls to more subsystems?

Exactly. Each time new framework will be introduced or reworked (or even for some drivers),
because each of them implement its own of_get_xxx() API. 

> 
>> - late_initcall is not (as for me not safe) a good time to register devices. A lot
>>    of platforms/subsystems/frameworks perform their final initialization or clean-up
>>    steps, with assumption that System mostly ready to work. For example, CPUIdle/CPUFreq
>>    are allowed and other PM staff. CPUIdle and driver's probing are not friends.
> 
> Yeah, I was expecting to find more problems due to this, but the
> platforms I tested on didn't show any. Do you have pointers to
> concrete issues?

No. This observation comes from my working experience with OMAP4 devices where
lowest possible CPUIdle state was mostly equal to Device-OFF state. 
Fast search has allowed me to find possible source of issues in code - I'm pretty sure
smth. similar can be found in other ARM maches:
- arch/arm/mach-omap2/omap_device.c -> omap_device_late_init().

> 
>> What would be nice to have for now in my opinion:
>> - some useful place to move device population code.
>>    May be machine_desc->init_device_sync() callback.
> 
> I'm looking at leave device registration where it currently is and
> just add devices to the deferred list until we get to late_initcall,
> where we would start to actually probe them. Seems promising for now.
> 
>> - some optional feature in DTC which will allow to re-arrange DT nodes for levels 0(root)
>>    and 1(simple-bus) using standard or widely used bindings (gpio, regulators, clocks, dma,
>>    pinctrl, irqs).
> 
> Could you extend on this, please?

This is actually mostly the same idea as was mentioned by Rob Clark 
http://www.spinics.net/lists/arm-kernel/msg423485.html


For example, mmc1 probe will be deferred always for below DT,
but if we put evm_3v3_sw before ocp and exchange mmc1 and i2c1
- mmc1 will be probed without deferring.
Potentially, It can be done in DTC where we do not have so strict limits
as for Kernel code. 

Soc file:
/ { /* root - level 0 */
	ocp { /* simple-bus - level 1 */
		compatible =  "simple-bus";

		mmc1: mmc@4809c000 {
			compatible = "mmc";
		...
		}

		i2c1: i2c@48070000 {
			compatible = "i2c";
		}
	}
}

board file:
#include "SoC.dtsi"

/ {
	evm_3v3_sw: fixedregulator-evm_3v3_sw {
		compatible = "regulator-fixed";
		regulator-name = "evm_3v3_sw";
		regulator-min-microvolt = <3300000>;
		regulator-max-microvolt = <3300000>;
	};
};

&i2c1 {
	pcf_gpio_21: gpio@21 {
		compatible = "ti,pcf8575";
		reg = <0x21>;
		lines-initial-states = <0x1408>;
		gpio-controller;
		#gpio-cells = <2>;
	};
}

&mmc1 {
	status = "okay";
	vmmc-supply = <&evm_3v3_sw>;
	bus-width = <4>;
	cd-gpios = <&pcf_gpio_21 5 0>;
};
	


-- 
regards,
-grygorii

Re: [PATCH 00/21] On-demand device registration

[Alexander Holler]

Am 03.06.2015 um 21:57 schrieb Grygorii.Strashko@linaro.org:

...

> So few comments from above:
> - registering devices later during the System boot may improve boot time.
>   But resolving of all deferred probes may NOT improve boot time ;) 
>   Have you seen smth like this?

If someone is out for boot time reduction, I think one of the best ways
would by making driver initialization parallel. Keep in mind that all
linked in drivers currently are initialized in series.

As it seems to have been forgotten or overread, I've mentioned in my
series of patches last year that, with a few changes, it's possible to
let the algorithm I've used (dfs) to spit out all drivers which can be
initialized in parallel.

But as I'm not paid for the work I've done and just did it out of
curiosity, interest or how ever you want name it, I haven't spend any
more time into that topic, especially as I'm missing the necessary
connections to get patches into the kernel. ;)

But, as said, it's easy (at least if aren't getting panic when it comes
to a bit of algorithm theory) to get a list drivers you can start in
parallel if you have such a complete list of dependencies as DT already
offers. Just look at the pictures generate by dtc (using my patches),
you will see, they already show which drivers can be initialized in
parallel.

So it would be easy to use e.g. all cores already very early at boot to
initialize drivers, not just after init got started. Besides that the
würgaround of defered init (which, btw. leads devs to supress error
messages, which is especially bad if you are searching a problem) isn't
needed anymore if you have a list of dependecies (however you get it,
I've used DT because the dependencies already are all there).

Regards,

Alexander Holler

Re: [PATCH 00/21] On-demand device registration

[Enrico Weigelt, metux IT consult]

Am 04.06.2015 um 22:39 schrieb Alexander Holler:

 > As it seems to have been forgotten or overread, I've mentioned in my
> series of patches last year that, with a few changes, it's possible to
> let the algorithm I've used (dfs) to spit out all drivers which can be
> initialized in parallel.

Unfortunately, I've missed that ... could you please resend you patches?
Boot time reduction is one of the topics on my 2do in several weeks.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.

Re: [PATCH 00/21] On-demand device registration

[Alexander Holler]

Am 08.06.2015 um 14:26 schrieb Enrico Weigelt, metux IT consult:
> Am 04.06.2015 um 22:39 schrieb Alexander Holler:
>
>  > As it seems to have been forgotten or overread, I've mentioned in my
>> series of patches last year that, with a few changes, it's possible to
>> let the algorithm I've used (dfs) to spit out all drivers which can be
>> initialized in parallel.
>
> Unfortunately, I've missed that ... could you please resend you patches?
> Boot time reduction is one of the topics on my 2do in several weeks.

https://lkml.org/lkml/2014/5/12/452

Re: [PATCH 00/21] On-demand device registration

[Alexander Holler]

Am 08.06.2015 um 20:14 schrieb Alexander Holler:
> Am 08.06.2015 um 14:26 schrieb Enrico Weigelt, metux IT consult:
>> Am 04.06.2015 um 22:39 schrieb Alexander Holler:
>>
>>  > As it seems to have been forgotten or overread, I've mentioned in my
>>> series of patches last year that, with a few changes, it's possible to
>>> let the algorithm I've used (dfs) to spit out all drivers which can be
>>> initialized in parallel.
>>
>> Unfortunately, I've missed that ... could you please resend you patches?
>> Boot time reduction is one of the topics on my 2do in several weeks.
>
> https://lkml.org/lkml/2014/5/12/452
>

And don't forget patch 10/9 which fixed a bug in my previous patch 
series and which alos was the reason for the large difference in boot 
times with and without deps:

https://lkml.org/lkml/2014/5/13/567

Re: [PATCH 00/21] On-demand device registration

[Tomeu Vizoso]

On 28 May 2015 at 06:33, Rob Herring <robherring2@gmail.com> wrote:
> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso
> <tomeu.vizoso@collabora.com> wrote:
>> Hello,
>>
>> I have a problem with the panel on my Tegra Chromebook taking longer than
>> expected to be ready during boot (Stéphane Marchesin reported what is
>> basically the same issue in [0]), and have looked into ordered probing as a
>> better way of solving this than moving nodes around in the DT or playing with
>> initcall levels.
>>
>> While reading the thread [1] that Alexander Holler started with his series to
>> make probing order deterministic, it occurred to me that it should be possible
>> to achieve the same by registering devices as they are referenced by other
>> devices.
>
> I like the concept and novel approach.
>
>> This basically reuses the information that is already implicit in the probe()
>> implementations, saving us from refactoring existing drivers or adding
>> information to DTBs.
>>
>> Something I'm not completely happy with is that I have had to move the call to
>> of_platform_populate after all platform drivers have been registered.
>> Otherwise I don't see how I could register drivers on demand as we don't have
>> yet each driver's compatible strings.
>
> Yeah, this is the opposite of what we'd really like.

Can you elaborate on the reasons why we would like to have devices
registered before built-in drivers finish registering, even if we
don't probe them yet?

> Ideally, we would
> have a solution that works for modules too. However, we're no worse
> off. We pretty much build-in dependencies to avoid module ordering
> problems.

Nod, I haven't looked yet at requesting modules on-demand, but I guess
it should be doable. Modules that have dependencies described in the
firmware should get them probed automatically already though.

> Perhaps we need to make the probing on-demand rather than simply on
> device<->driver match occurring.

I'm afraid that too much old code depends on that. For example, Rafael
pointed out to the PNP subsystem, which registers a driver that will
probe devices with the EISA ID PNP0c02 to reserve memory regions for
devices that will be probed later.

http://lxr.free-electrons.com/source/drivers/pnp/system.c

My understanding is that probing of PNP0c02 devices must happen before
the actual devices that depend on those regions are probed, so if we
decoupled the probing from the driver/device registration, we would be
breaking that assumption.

>> For machs that don't move of_platform_populate() to a later point, these
>> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
>> all the deferred probes as some drivers may not be registered yet.
>
> Ideally, of_platform_populate is not explicitly called by each
> platform. So I think we need to make this work for the default case.

The problem is that some platforms will need fixing because some
initcalls assume that some devices will have been registered already,
or even probed. I think removing those assumptions shouldn't be
problematic because I haven't had much trouble with this on the four
platforms I have tested with, but I cannot test every board that is
supported upstream.

I can ask though the KernelCI folks to boot my branch in all their
boards and make sure that those work when of_platform_populate is
called in late_initcall.

http://kernelci.org/boot/all/job/next/kernel/next-20150619/

>> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
>> patches were enough to eliminate all the deferred probes.
>>
>> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.
>
> That's certainly compelling.

Have to say that those numbers are with the serial console enabled
(without, it's 0.5s vs 1.5s), but on machines that take longer to boot
we should see bigger gains because we won't be sending devices to the
end of the queue when their probe is deferred.

Regards,

Tomeu

> Rob
>
>>
>> Regards,
>>
>> Tomeu
>>
>> [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html
>>
>> [1] https://lkml.org/lkml/2014/5/12/452
>>
>> Tomeu Vizoso (21):
>>   regulator: core: Reduce critical area in _regulator_get
>>   ARM: tegra: Add gpio-ranges property
>>   ARM: tegra: Register drivers before devices
>>   ARM: EXYNOS: Register drivers before devices
>>   ARM i.MX6q: Register drivers before devices
>>   of/platform: Add of_platform_device_ensure()
>>   of/platform: Ensure device registration on lookup
>>   gpio: Probe GPIO drivers on demand
>>   gpio: Probe pinctrl devices on demand
>>   regulator: core: Probe regulators on demand
>>   drm: Probe panels on demand
>>   drm/tegra: Probe dpaux devices on demand
>>   i2c: core: Probe i2c master devices on demand
>>   pwm: Probe PWM chip devices on demand
>>   backlight: Probe backlight devices on demand
>>   usb: phy: Probe phy devices on demand
>>   clk: Probe clk providers on demand
>>   pinctrl: Probe pinctrl devices on demand
>>   phy: core: Probe phy providers on demand
>>   dma: of: Probe DMA controllers on demand
>>   power-supply: Probe power supplies on demand
>>
>>  arch/arm/boot/dts/tegra124.dtsi     |  1 +
>>  arch/arm/mach-exynos/exynos.c       |  4 +--
>>  arch/arm/mach-imx/mach-imx6q.c      | 12 ++++-----
>>  arch/arm/mach-tegra/tegra.c         | 21 ++++++---------
>>  drivers/clk/clk.c                   |  3 +++
>>  drivers/dma/of-dma.c                |  3 +++
>>  drivers/gpio/gpiolib-of.c           |  5 ++++
>>  drivers/gpu/drm/drm_panel.c         |  3 +++
>>  drivers/gpu/drm/tegra/dpaux.c       |  3 +++
>>  drivers/i2c/i2c-core.c              |  3 +++
>>  drivers/of/platform.c               | 53 +++++++++++++++++++++++++++++++++++++
>>  drivers/phy/phy-core.c              |  3 +++
>>  drivers/pinctrl/devicetree.c        |  2 ++
>>  drivers/power/power_supply_core.c   |  3 +++
>>  drivers/pwm/core.c                  |  3 +++
>>  drivers/regulator/core.c            | 45 +++++++++++++++----------------
>>  drivers/usb/phy/phy.c               |  3 +++
>>  drivers/video/backlight/backlight.c |  3 +++
>>  include/linux/of_platform.h         |  2 ++
>>  19 files changed, 130 insertions(+), 45 deletions(-)
>>
>> --
>> 2.4.1
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in

Re: [PATCH 00/21] On-demand device registration

[Rob Herring]

On Mon, Jun 22, 2015 at 10:23 AM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
> On 28 May 2015 at 06:33, Rob Herring <robherring2@gmail.com> wrote:
>> On Mon, May 25, 2015 at 9:53 AM, Tomeu Vizoso
>> <tomeu.vizoso@collabora.com> wrote:
>>> Hello,
>>>
>>> I have a problem with the panel on my Tegra Chromebook taking longer than
>>> expected to be ready during boot (Stéphane Marchesin reported what is
>>> basically the same issue in [0]), and have looked into ordered probing as a
>>> better way of solving this than moving nodes around in the DT or playing with
>>> initcall levels.
>>>
>>> While reading the thread [1] that Alexander Holler started with his series to
>>> make probing order deterministic, it occurred to me that it should be possible
>>> to achieve the same by registering devices as they are referenced by other
>>> devices.
>>
>> I like the concept and novel approach.
>>
>>> This basically reuses the information that is already implicit in the probe()
>>> implementations, saving us from refactoring existing drivers or adding
>>> information to DTBs.
>>>
>>> Something I'm not completely happy with is that I have had to move the call to
>>> of_platform_populate after all platform drivers have been registered.
>>> Otherwise I don't see how I could register drivers on demand as we don't have
>>> yet each driver's compatible strings.
>>
>> Yeah, this is the opposite of what we'd really like.
>
> Can you elaborate on the reasons why we would like to have devices
> registered before built-in drivers finish registering, even if we
> don't probe them yet?

My main thought was for modules we will almost always have devices
appearing first. More generally, we can have devices and drivers
coming or going at any point in time and should not put restrictions
on ordering.

Also, I think all the probe ordering and dependency tracking should be
done within the driver core (i.e. dependencies are a list of struct
devices). At some level it has to become firmware specific, but we
want to minimize that part. I could be convinced otherwise and you
have put more thought into this problem than I have.

>> Ideally, we would
>> have a solution that works for modules too. However, we're no worse
>> off. We pretty much build-in dependencies to avoid module ordering
>> problems.
>
> Nod, I haven't looked yet at requesting modules on-demand, but I guess
> it should be doable. Modules that have dependencies described in the
> firmware should get them probed automatically already though.
>
>> Perhaps we need to make the probing on-demand rather than simply on
>> device<->driver match occurring.
>
> I'm afraid that too much old code depends on that. For example, Rafael
> pointed out to the PNP subsystem, which registers a driver that will
> probe devices with the EISA ID PNP0c02 to reserve memory regions for
> devices that will be probed later.
>
> http://lxr.free-electrons.com/source/drivers/pnp/system.c
>
> My understanding is that probing of PNP0c02 devices must happen before
> the actual devices that depend on those regions are probed, so if we
> decoupled the probing from the driver/device registration, we would be
> breaking that assumption.

That shouldn't matter as PNP matching is PNP specific. We already have
different ways of matching with device/driver name or of_match_table
for example. Changing how and when OF matching occurs would not affect
PNP matching. We do matching on device and driver add currently. For
the "when" part, we would need to add what I'll call async matching or
deferred matching which in addition to matching on the of_match_table
also matches on the dependency list having probed. Your last series
essentially does this, but the difference is yours is not OF specific
and I think it needs to be. I mean it is OF specific only in the
aspect that matching already is. From a driver and subsystem
standpoint, it should not be OF specific much like deferred probe is
not OF specific, but in reality only occurs (currently) on OF probed
drivers.

>>> For machs that don't move of_platform_populate() to a later point, these
>>> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
>>> all the deferred probes as some drivers may not be registered yet.
>>
>> Ideally, of_platform_populate is not explicitly called by each
>> platform. So I think we need to make this work for the default case.
>
> The problem is that some platforms will need fixing because some
> initcalls assume that some devices will have been registered already,
> or even probed. I think removing those assumptions shouldn't be
> problematic because I haven't had much trouble with this on the four
> platforms I have tested with, but I cannot test every board that is
> supported upstream.
>
> I can ask though the KernelCI folks to boot my branch in all their
> boards and make sure that those work when of_platform_populate is
> called in late_initcall.

I'd imagine Kevin would be happy to. That is still a subset of h/w, so
we'd need a way to disable any solution.

Rob
--
To unsubscribe from this list: send the line "unsubscribe linux-clk" in

Re: [PATCH 00/21] On-demand device registration

[Linus Walleij]

On Mon, May 25, 2015 at 4:53 PM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:

> have looked into ordered probing as a
> better way of solving this than moving nodes around in the DT or playing with
> initcall levels.
>
> While reading the thread [1] that Alexander Holler started with his series to
> make probing order deterministic, it occurred to me that it should be possible
> to achieve the same by registering devices as they are referenced by other
> devices.

This is pretty cool, but a too local solution to a global problem.

Deferred probe and initcall reordering, silly as they may seem,
does not require you to use device tree.

The real solution, which I think I pointed out already when we
added deferred probe, is to put dependency graphs in the drivers
and have the kernel device driver core percolate dependecies by
walking the graph on probing driver, removing driver (usually the
inverse use case), [runtime] suspend and [runtime] resumeing
a driver. Possibly the dependencies will even be different
depending on use case.

This is what systemd is doing in userspace for starting services:
ask for your dependencies and wait for them if they are not
there. So drivers ask for resources and wait for them. It also
needs to be abstract, so for example we need to be able to
hang on regulator_get() until the driver is up and providing that
regulator, and as long as everything is in slowpath it should
be OK. (And vice versa mutatis mutandis for clk, gpio, pin
control, interrupts (!) and DMA channels for example.)


So if this should be solved it should be solved in an abstract way
in the device driver core available for all, then have calls calling
out to DT, ACPI, possibly even PCI or USB (as these
enumerate devices themselves) to obtain a certain
dependency.

Yours,
Linus Walleij

Re: [PATCH 00/21] On-demand device registration

[Tomeu Vizoso]

On 2 June 2015 at 10:48, Linus Walleij <linus.walleij@linaro.org> wrote:
> On Mon, May 25, 2015 at 4:53 PM, Tomeu Vizoso
> <tomeu.vizoso@collabora.com> wrote:
>
>> have looked into ordered probing as a
>> better way of solving this than moving nodes around in the DT or playing with
>> initcall levels.
>>
>> While reading the thread [1] that Alexander Holler started with his series to
>> make probing order deterministic, it occurred to me that it should be possible
>> to achieve the same by registering devices as they are referenced by other
>> devices.
>
> This is pretty cool, but a too local solution to a global problem.
>
> Deferred probe and initcall reordering, silly as they may seem,
> does not require you to use device tree.
>
> The real solution, which I think I pointed out already when we
> added deferred probe, is to put dependency graphs in the drivers

By this you mean something like what Thierry suggested here?

http://article.gmane.org/gmane.linux.kernel/1774623

> and have the kernel device driver core percolate dependecies by
> walking the graph on probing driver, removing driver (usually the
> inverse use case), [runtime] suspend and [runtime] resumeing
> a driver. Possibly the dependencies will even be different
> depending on use case.
>
> This is what systemd is doing in userspace for starting services:
> ask for your dependencies and wait for them if they are not
> there. So drivers ask for resources and wait for them. It also
> needs to be abstract, so for example we need to be able to
> hang on regulator_get() until the driver is up and providing that
> regulator, and as long as everything is in slowpath it should
> be OK. (And vice versa mutatis mutandis for clk, gpio, pin
> control, interrupts (!) and DMA channels for example.)

I understood above that you propose probing devices in order, but now
you mention that resource getters would block until the dependency is
fulfilled which confuses me because if we are probing in order then
all dependencies would be fulfilled before the device in question gets
probed.

> So if this should be solved it should be solved in an abstract way
> in the device driver core available for all, then have calls calling
> out to DT, ACPI, possibly even PCI or USB (as these
> enumerate devices themselves) to obtain a certain
> dependency.

Yeah, I was planning looking into this now that I got it working with
async probing.

Thanks,

Tomeu

> Yours,
> Linus Walleij
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 00/21] On-demand device registration

[Linus Walleij]

On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
> On 2 June 2015 at 10:48, Linus Walleij <linus.walleij@linaro.org> wrote:

>> This is what systemd is doing in userspace for starting services:
>> ask for your dependencies and wait for them if they are not
>> there. So drivers ask for resources and wait for them. It also
>> needs to be abstract, so for example we need to be able to
>> hang on regulator_get() until the driver is up and providing that
>> regulator, and as long as everything is in slowpath it should
>> be OK. (And vice versa mutatis mutandis for clk, gpio, pin
>> control, interrupts (!) and DMA channels for example.)
>
> I understood above that you propose probing devices in order, but now
> you mention that resource getters would block until the dependency is
> fulfilled which confuses me because if we are probing in order then
> all dependencies would be fulfilled before the device in question gets
> probed.

Sorry, the problem space is a bit convoluted so the answers
get a bit convoluted. Maybe I'm thinking aloud and altering the course
of my thoughts as I type...

I guess there can be explicit dependencies for resources like this
patch does, but another way would be for all resource fetch functions
to be instrumented, so that you do not block until you try to take
a resource that is not yet there, e.g.:

regulator_get(...) -> not available, so:
- identify target regulator provider - this will need instrumentation
- probe it

It then turns out the regulator driver is on the i2c bus, so we
need to probe the i2c driver:
- identify target i2c host for the regulator driver - this will need
  instrumentation
- probe the i2c host driver

i2c host comes out, probes the regulator driver, regulator driver
probes and then the regulator_get() call returns.

This requires instrumentation on anything providing a resource
to another driver like those I mentioned and a lot of overhead
infrastructure, but I think it's the right approach. However I don't
know if I would ever be able to pull that off myself, I know talk
is cheap and I should show the code instead.

Deepest respect for your efforts!

Yours,
Linus Walleij

Re: [PATCH 00/21] On-demand device registration

[Tomeu Vizoso]

On 10 June 2015 at 09:30, Linus Walleij <linus.walleij@linaro.org> wrote:
> On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso
> <tomeu.vizoso@collabora.com> wrote:
>> On 2 June 2015 at 10:48, Linus Walleij <linus.walleij@linaro.org> wrote:
>
>>> This is what systemd is doing in userspace for starting services:
>>> ask for your dependencies and wait for them if they are not
>>> there. So drivers ask for resources and wait for them. It also
>>> needs to be abstract, so for example we need to be able to
>>> hang on regulator_get() until the driver is up and providing that
>>> regulator, and as long as everything is in slowpath it should
>>> be OK. (And vice versa mutatis mutandis for clk, gpio, pin
>>> control, interrupts (!) and DMA channels for example.)
>>
>> I understood above that you propose probing devices in order, but now
>> you mention that resource getters would block until the dependency is
>> fulfilled which confuses me because if we are probing in order then
>> all dependencies would be fulfilled before the device in question gets
>> probed.
>
> Sorry, the problem space is a bit convoluted so the answers
> get a bit convoluted. Maybe I'm thinking aloud and altering the course
> of my thoughts as I type...
>
> I guess there can be explicit dependencies for resources like this
> patch does, but another way would be for all resource fetch functions
> to be instrumented, so that you do not block until you try to take
> a resource that is not yet there, e.g.:
>
> regulator_get(...) -> not available, so:
> - identify target regulator provider - this will need instrumentation
> - probe it
>
> It then turns out the regulator driver is on the i2c bus, so we
> need to probe the i2c driver:
> - identify target i2c host for the regulator driver - this will need
>   instrumentation
> - probe the i2c host driver
>
> i2c host comes out, probes the regulator driver, regulator driver
> probes and then the regulator_get() call returns.

Hmm, if I understand correctly what you say, this is exactly what this
particular series does:

regulator_get -> of_platform_device_ensure -> probe() on the platform
device that encloses the requested device node (i2c host) -> i2c slave
gets probed and the regulator registered -> regulator_get returns the
requested resource

The downside I'm currently looking at is that an explicit dependency
graph would be useful to have for other purposes. For example to print
a neat warning when a dependency cannot be fulfilled. Or to refuse to
unbind a device which other devices depend on, or to automatically
unbind the devices that depend on it, or to print a warning if a
device is hotplugged off and other devices depend on it.

> This requires instrumentation on anything providing a resource
> to another driver like those I mentioned and a lot of overhead
> infrastructure, but I think it's the right approach. However I don't
> know if I would ever be able to pull that off myself, I know talk
> is cheap and I should show the code instead.

Yeah, if you can give it a second look and say if it matches what you
wrote above, it would be very much appreciated.

> Deepest respect for your efforts!

Thanks!

Tomeu

> Yours,
> Linus Walleij
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 00/21] On-demand device registration

[Andrzej Hajda]

On 06/10/2015 12:19 PM, Tomeu Vizoso wrote:
> On 10 June 2015 at 09:30, Linus Walleij <linus.walleij@linaro.org> wrote:
>> On Tue, Jun 2, 2015 at 12:14 PM, Tomeu Vizoso
>> <tomeu.vizoso@collabora.com> wrote:
>>> On 2 June 2015 at 10:48, Linus Walleij <linus.walleij@linaro.org> wrote:
>>
>>>> This is what systemd is doing in userspace for starting services:
>>>> ask for your dependencies and wait for them if they are not
>>>> there. So drivers ask for resources and wait for them. It also
>>>> needs to be abstract, so for example we need to be able to
>>>> hang on regulator_get() until the driver is up and providing that
>>>> regulator, and as long as everything is in slowpath it should
>>>> be OK. (And vice versa mutatis mutandis for clk, gpio, pin
>>>> control, interrupts (!) and DMA channels for example.)
>>>
>>> I understood above that you propose probing devices in order, but now
>>> you mention that resource getters would block until the dependency is
>>> fulfilled which confuses me because if we are probing in order then
>>> all dependencies would be fulfilled before the device in question gets
>>> probed.
>>
>> Sorry, the problem space is a bit convoluted so the answers
>> get a bit convoluted. Maybe I'm thinking aloud and altering the course
>> of my thoughts as I type...
>>
>> I guess there can be explicit dependencies for resources like this
>> patch does, but another way would be for all resource fetch functions
>> to be instrumented, so that you do not block until you try to take
>> a resource that is not yet there, e.g.:
>>
>> regulator_get(...) -> not available, so:
>> - identify target regulator provider - this will need instrumentation
>> - probe it
>>
>> It then turns out the regulator driver is on the i2c bus, so we
>> need to probe the i2c driver:
>> - identify target i2c host for the regulator driver - this will need
>>   instrumentation
>> - probe the i2c host driver
>>
>> i2c host comes out, probes the regulator driver, regulator driver
>> probes and then the regulator_get() call returns.
> 
> Hmm, if I understand correctly what you say, this is exactly what this
> particular series does:
> 
> regulator_get -> of_platform_device_ensure -> probe() on the platform
> device that encloses the requested device node (i2c host) -> i2c slave
> gets probed and the regulator registered -> regulator_get returns the
> requested resource

The downside of this solution is that it will not work without device
tree or even without device dependencies not explicitly specified in
device tree.

> 
> The downside I'm currently looking at is that an explicit dependency
> graph would be useful to have for other purposes. For example to print
> a neat warning when a dependency cannot be fulfilled. Or to refuse to
> unbind a device which other devices depend on,

As I understand Greg you cannot prevent unbinding by design, see [1].

[1]: http://thread.gmane.org/gmane.linux.kernel/1154308/focus=1154648

> or to automatically
> unbind the devices that depend on it,

What about devices that have weak dependency? They should not be unbound
but they should be somehow noticed about unbinding.

In general many kernel frameworks are broken in handling hot-unbinding
of drivers, consumers are not noticed about unbinding of their resource
providers and usually they stay with broken handles or handles to dummy
resources.

I suspect the only proper solution for handling resources that can
dynamically appear/disappear is to provide notification to their
consumers about appearance change of the resource.

I have proposed some times ago solution for above problems based on the
statement above, cover letter explains it in more detail [2].
In short it solves following issues:
- consumer receives resource as soon as it becomes available,
- consumer is notified just before resource removal,
- it can properly handle provider unbind/re-bind,
- it avoids late init due to deferred probing,
- it allows to track optional resources.

[2]: http://thread.gmane.org/gmane.linux.kernel.gpio/5201

Regards
Andrzej

> or to print a warning if a
> device is hotplugged off and other devices depend on it.
> 
>> This requires instrumentation on anything providing a resource
>> to another driver like those I mentioned and a lot of overhead
>> infrastructure, but I think it's the right approach. However I don't
>> know if I would ever be able to pull that off myself, I know talk
>> is cheap and I should show the code instead.
> 
> Yeah, if you can give it a second look and say if it matches what you
> wrote above, it would be very much appreciated.
> 
>> Deepest respect for your efforts!
> 
> Thanks!
> 
> Tomeu
> 
>> Yours,
>> Linus Walleij
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/dri-devel
> --
> To unsubscribe from this list: send the line "unsubscribe linux-gpio" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 00/21] On-demand device registration

[Linus Walleij]

On Wed, Jun 10, 2015 at 12:19 PM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
> On 10 June 2015 at 09:30, Linus Walleij <linus.walleij@linaro.org> wrote:

>> regulator_get(...) -> not available, so:
>> - identify target regulator provider - this will need instrumentation
>> - probe it
>>
>> It then turns out the regulator driver is on the i2c bus, so we
>> need to probe the i2c driver:
>> - identify target i2c host for the regulator driver - this will need
>>   instrumentation
>> - probe the i2c host driver
>>
>> i2c host comes out, probes the regulator driver, regulator driver
>> probes and then the regulator_get() call returns.
>
> Hmm, if I understand correctly what you say, this is exactly what this
> particular series does:
>
> regulator_get -> of_platform_device_ensure -> probe() on the platform
> device that encloses the requested device node (i2c host) -> i2c slave
> gets probed and the regulator registered -> regulator_get returns the
> requested resource

Yes. But only for device tree.

> The downside I'm currently looking at is that an explicit dependency
> graph would be useful to have for other purposes. For example to print
> a neat warning when a dependency cannot be fulfilled. Or to refuse to
> unbind a device which other devices depend on, or to automatically
> unbind the devices that depend on it, or to print a warning if a
> device is hotplugged off and other devices depend on it.

Unbind/remove() calls are the inverse usually yes.

But also the [runtime] power up/down sequences for the
devices tend to depend on a similar ordering or mostly
the same. (Mentioned this before I think.)

>> This requires instrumentation on anything providing a resource
>> to another driver like those I mentioned and a lot of overhead
>> infrastructure, but I think it's the right approach. However I don't
>> know if I would ever be able to pull that off myself, I know talk
>> is cheap and I should show the code instead.
>
> Yeah, if you can give it a second look and say if it matches what you
> wrote above, it would be very much appreciated.

Yes you are right. But what about ACPI, board files,
Simple Firmware and future hardware description languages...

Yours,
Linus Walleij

Re: [PATCH 00/21] On-demand device registration

[Tomeu Vizoso]

On 06/11/2015 10:15 AM, Linus Walleij wrote:
> On Wed, Jun 10, 2015 at 12:19 PM, Tomeu Vizoso
> <tomeu.vizoso@collabora.com> wrote:
>> On 10 June 2015 at 09:30, Linus Walleij <linus.walleij@linaro.org> wrote:
> 
>>> regulator_get(...) -> not available, so:
>>> - identify target regulator provider - this will need instrumentation
>>> - probe it
>>>
>>> It then turns out the regulator driver is on the i2c bus, so we
>>> need to probe the i2c driver:
>>> - identify target i2c host for the regulator driver - this will need
>>>   instrumentation
>>> - probe the i2c host driver
>>>
>>> i2c host comes out, probes the regulator driver, regulator driver
>>> probes and then the regulator_get() call returns.
>>
>> Hmm, if I understand correctly what you say, this is exactly what this
>> particular series does:
>>
>> regulator_get -> of_platform_device_ensure -> probe() on the platform
>> device that encloses the requested device node (i2c host) -> i2c slave
>> gets probed and the regulator registered -> regulator_get returns the
>> requested resource
> 
> Yes. But only for device tree.
> 
>> The downside I'm currently looking at is that an explicit dependency
>> graph would be useful to have for other purposes. For example to print
>> a neat warning when a dependency cannot be fulfilled. Or to refuse to
>> unbind a device which other devices depend on, or to automatically
>> unbind the devices that depend on it, or to print a warning if a
>> device is hotplugged off and other devices depend on it.
> 
> Unbind/remove() calls are the inverse usually yes.
> 
> But also the [runtime] power up/down sequences for the
> devices tend to depend on a similar ordering or mostly
> the same. (Mentioned this before I think.)
> 
>>> This requires instrumentation on anything providing a resource
>>> to another driver like those I mentioned and a lot of overhead
>>> infrastructure, but I think it's the right approach. However I don't
>>> know if I would ever be able to pull that off myself, I know talk
>>> is cheap and I should show the code instead.
>>
>> Yeah, if you can give it a second look and say if it matches what you
>> wrote above, it would be very much appreciated.
> 
> Yes you are right. But what about ACPI, board files,
> Simple Firmware and future hardware description languages...

Ah ok, got it now. With fwnode and by moving a bit of code around that
shouldn't be a problem.

I'm actually now implementing the alternative approach in which
dependencies are discovered before the device is probed, then probed in
turn until all are available. So functionally is very similar but I
expect to find big differences in how the codebase is impacted.

Regards,

Tomeu

> Yours,
> Linus Walleij
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 00/21] On-demand device registration

[Rob Clark]

On Mon, May 25, 2015 at 10:53 AM, Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
> Hello,
>
> I have a problem with the panel on my Tegra Chromebook taking longer than
> expected to be ready during boot (Stéphane Marchesin reported what is
> basically the same issue in [0]), and have looked into ordered probing as a
> better way of solving this than moving nodes around in the DT or playing with
> initcall levels.
>
> While reading the thread [1] that Alexander Holler started with his series to
> make probing order deterministic, it occurred to me that it should be possible
> to achieve the same by registering devices as they are referenced by other
> devices.
>
> This basically reuses the information that is already implicit in the probe()
> implementations, saving us from refactoring existing drivers or adding
> information to DTBs.
>
> Something I'm not completely happy with is that I have had to move the call to
> of_platform_populate after all platform drivers have been registered.
> Otherwise I don't see how I could register drivers on demand as we don't have
> yet each driver's compatible strings.
>
> For machs that don't move of_platform_populate() to a later point, these
> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
> all the deferred probes as some drivers may not be registered yet.
>
> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
> patches were enough to eliminate all the deferred probes.
>
> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.

So, complete drive-by comment (and I won't claim to be a DT expert,
etc, etc, so take this with a few grains of salt), but why not push
the problem to the DT compiler (or a pre-process step that could be
run on existing DT blobs), which generates an optional DT node that is
the recommended probe order?  That seems like it avoids adding
complexity into the early boot code (which seems like a good thing)..

As a bonus, a bootarg (or something like that) which runs through the
recommended probe order in reverse (to continue our current state of
ensuring that EPROBE_DEFER error paths are well tested)

At any rate, for a device like a drm driver that has multiple
sub-components, and depends on various other clk/gpio/regulator/etc
drivers, the current EPROBE_DEFER situation is pretty comical, so any
solution that improves on things is very much welcome :-)

BR,
-R

> Regards,
>
> Tomeu
>
> [0] http://lists.freedesktop.org/archives/dri-devel/2014-August/066527.html
>
> [1] https://lkml.org/lkml/2014/5/12/452
>
> Tomeu Vizoso (21):
>   regulator: core: Reduce critical area in _regulator_get
>   ARM: tegra: Add gpio-ranges property
>   ARM: tegra: Register drivers before devices
>   ARM: EXYNOS: Register drivers before devices
>   ARM i.MX6q: Register drivers before devices
>   of/platform: Add of_platform_device_ensure()
>   of/platform: Ensure device registration on lookup
>   gpio: Probe GPIO drivers on demand
>   gpio: Probe pinctrl devices on demand
>   regulator: core: Probe regulators on demand
>   drm: Probe panels on demand
>   drm/tegra: Probe dpaux devices on demand
>   i2c: core: Probe i2c master devices on demand
>   pwm: Probe PWM chip devices on demand
>   backlight: Probe backlight devices on demand
>   usb: phy: Probe phy devices on demand
>   clk: Probe clk providers on demand
>   pinctrl: Probe pinctrl devices on demand
>   phy: core: Probe phy providers on demand
>   dma: of: Probe DMA controllers on demand
>   power-supply: Probe power supplies on demand
>
>  arch/arm/boot/dts/tegra124.dtsi     |  1 +
>  arch/arm/mach-exynos/exynos.c       |  4 +--
>  arch/arm/mach-imx/mach-imx6q.c      | 12 ++++-----
>  arch/arm/mach-tegra/tegra.c         | 21 ++++++---------
>  drivers/clk/clk.c                   |  3 +++
>  drivers/dma/of-dma.c                |  3 +++
>  drivers/gpio/gpiolib-of.c           |  5 ++++
>  drivers/gpu/drm/drm_panel.c         |  3 +++
>  drivers/gpu/drm/tegra/dpaux.c       |  3 +++
>  drivers/i2c/i2c-core.c              |  3 +++
>  drivers/of/platform.c               | 53 +++++++++++++++++++++++++++++++++++++
>  drivers/phy/phy-core.c              |  3 +++
>  drivers/pinctrl/devicetree.c        |  2 ++
>  drivers/power/power_supply_core.c   |  3 +++
>  drivers/pwm/core.c                  |  3 +++
>  drivers/regulator/core.c            | 45 +++++++++++++++----------------
>  drivers/usb/phy/phy.c               |  3 +++
>  drivers/video/backlight/backlight.c |  3 +++
>  include/linux/of_platform.h         |  2 ++
>  19 files changed, 130 insertions(+), 45 deletions(-)
>
> --
> 2.4.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 00/21] On-demand device registration

[Alexander Holler]

Am 03.06.2015 um 23:12 schrieb Rob Clark:
> On Mon, May 25, 2015 at 10:53 AM, Tomeu Vizoso
> <tomeu.vizoso@collabora.com> wrote:
>> Hello,
>>
>> I have a problem with the panel on my Tegra Chromebook taking longer than
>> expected to be ready during boot (Stéphane Marchesin reported what is
>> basically the same issue in [0]), and have looked into ordered probing as a
>> better way of solving this than moving nodes around in the DT or playing with
>> initcall levels.
>>
>> While reading the thread [1] that Alexander Holler started with his series to
>> make probing order deterministic, it occurred to me that it should be possible
>> to achieve the same by registering devices as they are referenced by other
>> devices.
>>
>> This basically reuses the information that is already implicit in the probe()
>> implementations, saving us from refactoring existing drivers or adding
>> information to DTBs.
>>
>> Something I'm not completely happy with is that I have had to move the call to
>> of_platform_populate after all platform drivers have been registered.
>> Otherwise I don't see how I could register drivers on demand as we don't have
>> yet each driver's compatible strings.
>>
>> For machs that don't move of_platform_populate() to a later point, these
>> patches shouldn't cause any problems but it's not guaranteed that we'll avoid
>> all the deferred probes as some drivers may not be registered yet.
>>
>> I have tested this on boards with Tegra, iMX.6 and Exynos SoCs, and these
>> patches were enough to eliminate all the deferred probes.
>>
>> With this series I get the kernel to output to the panel in 0.5s, instead of 2.8s.
> 
> So, complete drive-by comment (and I won't claim to be a DT expert,
> etc, etc, so take this with a few grains of salt), but why not push
> the problem to the DT compiler (or a pre-process step that could be
> run on existing DT blobs), which generates an optional DT node that is
> the recommended probe order?  That seems like it avoids adding
> complexity into the early boot code (which seems like a good thing)..

I've played with that approach too (as my patches for dtc do contain the
same code I've put into the kernel, but decided that it doesn't make
much sense. The sort algorithm is really small (some dozen lines), very
fast (around 3-5ms on a omap) and might be later used to sort necessary
module loading too. So there would be no advantage to put a sorted list
into the DT. And having the sort algorithm in the kernel, would make it
possible to use it for acpi or something else too, if they manage it to
provide the necessary dependencies.

Regards,

Alexander Holler