[RFC 00/12] On-demand device registration

[RFC 00/12] On-demand device registration

[Tomeu Vizoso]

Hi,

while reading the thread [0] that Alexander Holler started with his series to make probing order deterministic, it occurred to me that it should be possible to achieve the same by probing devices as they are referenced by other devices.

This basically reuses the information that is already embedded in the probe() implementations, saving us from refactoring existing drivers or adding information to DTBs.

The main issue I see is that the registration code path in some subsystems may not be reentrant, so some refactoring of the locking will be needed. In my testing I have found this problem with regulators, as the supply of a regulator might end up being registered during the registration of the first one.

Something I'm not completely happy with is that I have had to move the population of the device tree after all platform drivers have been registered. Otherwise I don't see how I could register drivers on demand as we don't have yet each driver's compatible strings.

I have done my testing on a Tegra124-based Chromebook, and these patches were enough to eliminate all the deferred probes.

Regards,

Tomeu

[0] https://lkml.org/lkml/2014/5/12/452

Tomeu Vizoso (12):
  ARM: tegra: Register drivers before devices
  ARM: tegra: Add gpio-ranges property
  of/platform: add of_platform_device_ensure()
  gpio: Probe GPIO drivers on demand
  gpio: Probe pinctrl devices on demand
  regulator: core: Probe regulators on demand
  drm: Probe panels on demand
  drm/tegra: Probe dpaux devices on demand
  i2c: core: Probe i2c master devices on demand
  pwm: Probe PWM chip devices on demand
  backlight: Probe backlight devices on demand
  usb: phy: Probe phy devices on demand

 arch/arm/boot/dts/tegra124.dtsi     |  1 +
 arch/arm/mach-tegra/tegra.c         | 21 ++++++++-------------
 drivers/gpio/gpiolib-of.c           |  5 +++++
 drivers/gpu/drm/drm_panel.c         |  3 +++
 drivers/gpu/drm/tegra/dpaux.c       |  3 +++
 drivers/i2c/i2c-core.c              |  3 +++
 drivers/of/platform.c               | 28 ++++++++++++++++++++++++++++
 drivers/pwm/core.c                  |  3 +++
 drivers/regulator/core.c            |  2 ++
 drivers/usb/phy/phy.c               |  3 +++
 drivers/video/backlight/backlight.c |  3 +++
 include/linux/of_platform.h         |  2 ++
 12 files changed, 64 insertions(+), 13 deletions(-)

-- 
2.3.6

[RFC 02/12] ARM: tegra: Add gpio-ranges property

[Tomeu Vizoso]

Specify how the GPIOs map to the pins in T124, so the dependency is
explicit.
---
 arch/arm/boot/dts/tegra124.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/boot/dts/tegra124.dtsi b/arch/arm/boot/dts/tegra124.dtsi
index 13cc7ca..5d1d35f 100644
--- a/arch/arm/boot/dts/tegra124.dtsi
+++ b/arch/arm/boot/dts/tegra124.dtsi
@@ -254,6 +254,7 @@
 		gpio-controller;
 		#interrupt-cells = <2>;
 		interrupt-controller;
+		gpio-ranges = <&pinmux 0 0 250>;
 	};
 
 	apbdma: dma@0,60020000 {
-- 
2.3.6

[RFC 03/12] of/platform: add of_platform_device_ensure()

[Tomeu Vizoso]

This function ensures that the device that encloses the passed device
node is registered, and thus probed if the corresponding driver has been
registered already.

This function can be used by drivers to ensure that a dependency is
fulfilled.
---
 drivers/of/platform.c       | 28 ++++++++++++++++++++++++++++
 include/linux/of_platform.h |  2 ++
 2 files changed, 30 insertions(+)

diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index a01f57c..a71c36e 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -214,6 +214,34 @@ struct platform_device *of_platform_device_create(struct device_node *np,
 }
 EXPORT_SYMBOL(of_platform_device_create);
 
+/**
+ * of_platform_device_ensure - Ensure that device has been registered
+ * @np: pointer to node to create device for
+ *
+ * Ensures that a device has been registered for the given node.
+ */
+void of_platform_device_ensure(struct device_node *np)
+{
+	struct device_node *node;
+	struct platform_device *pdev;
+
+	/* Get the device that contains this node */
+	for(node = np;
+	    node && !of_get_property(node, "compatible", NULL);
+	    node = node->parent);
+
+	/* If it's in a non-simple bus, probe that one instead */
+	if (node->parent &&
+	    !of_node_is_root(node->parent) &&
+	    !of_match_node(of_default_bus_match_table, node->parent))
+		node = node->parent;
+
+	pdev = of_platform_device_create(node, NULL, NULL);
+	if (IS_ERR(pdev))
+		pr_debug("%s: creation of platform device failed\n", np->full_name);
+}
+EXPORT_SYMBOL_GPL(of_platform_device_ensure);
+
 #ifdef CONFIG_ARM_AMBA
 static struct amba_device *of_amba_device_create(struct device_node *node,
 						 const char *bus_id,
diff --git a/include/linux/of_platform.h b/include/linux/of_platform.h
index 611a691..ee6fa45 100644
--- a/include/linux/of_platform.h
+++ b/include/linux/of_platform.h
@@ -67,6 +67,8 @@ extern struct platform_device *of_platform_device_create(struct device_node *np,
 extern int of_platform_bus_probe(struct device_node *root,
 				 const struct of_device_id *matches,
 				 struct device *parent);
+
+extern void of_platform_device_ensure(struct device_node *np);
 #ifdef CONFIG_OF_ADDRESS
 extern int of_platform_populate(struct device_node *root,
 				const struct of_device_id *matches,
-- 
2.3.6

[RFC 01/12] ARM: tegra: Register drivers before devices

[Tomeu Vizoso]

So devices can be probed on demand, we need to have the drivers already
registered as we don't have enough information to register a driver on
demand.
---
 arch/arm/mach-tegra/tegra.c | 21 ++++++++-------------
 1 file changed, 8 insertions(+), 13 deletions(-)

diff --git a/arch/arm/mach-tegra/tegra.c b/arch/arm/mach-tegra/tegra.c
index 2378fa56..5fe5bd4 100644
--- a/arch/arm/mach-tegra/tegra.c
+++ b/arch/arm/mach-tegra/tegra.c
@@ -55,6 +55,8 @@
 #include "reset.h"
 #include "sleep.h"
 
+struct soc_device *soc_dev;
+
 /*
  * Storage for debug-macro.S's state.
  *
@@ -87,12 +89,10 @@ static void __init tegra_dt_init_irq(void)
 static void __init tegra_dt_init(void)
 {
 	struct soc_device_attribute *soc_dev_attr;
-	struct soc_device *soc_dev;
-	struct device *parent = NULL;
 
 	soc_dev_attr = kzalloc(sizeof(*soc_dev_attr), GFP_KERNEL);
 	if (!soc_dev_attr)
-		goto out;
+		return;
 
 	soc_dev_attr->family = kasprintf(GFP_KERNEL, "Tegra");
 	soc_dev_attr->revision = kasprintf(GFP_KERNEL, "%d",
@@ -105,17 +105,7 @@ static void __init tegra_dt_init(void)
 		kfree(soc_dev_attr->revision);
 		kfree(soc_dev_attr->soc_id);
 		kfree(soc_dev_attr);
-		goto out;
 	}
-
-	parent = soc_device_to_device(soc_dev);
-
-	/*
-	 * Finished with the static registrations now; fill in the missing
-	 * devices
-	 */
-out:
-	of_platform_populate(NULL, of_default_bus_match_table, NULL, parent);
 }
 
 static void __init paz00_init(void)
@@ -133,8 +123,13 @@ static struct {
 
 static void __init tegra_dt_init_late(void)
 {
+	struct device *parent = NULL;
 	int i;
 
+	parent = soc_device_to_device(soc_dev);
+
+	of_platform_populate(NULL, of_default_bus_match_table, NULL, parent);
+
 	tegra_init_suspend();
 	tegra_cpuidle_init();
 
-- 
2.3.6

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC 04/12] gpio: Probe GPIO drivers on demand

[Tomeu Vizoso]

Ensure that the device corresponding to the DT node of the gpiochip has
been registered.
---
 drivers/gpio/gpiolib-of.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
index a6c67c6..190183d 100644
--- a/drivers/gpio/gpiolib-of.c
+++ b/drivers/gpio/gpiolib-of.c
@@ -20,6 +20,7 @@
 #include <linux/of.h>
 #include <linux/of_address.h>
 #include <linux/of_gpio.h>
+#include <linux/of_platform.h>
 #include <linux/pinctrl/pinctrl.h>
 #include <linux/slab.h>
 #include <linux/gpio/machine.h>
@@ -95,6 +96,8 @@ struct gpio_desc *of_get_named_gpiod_flags(struct device_node *np,
 		return ERR_PTR(ret);
 	}
 
+	of_platform_device_ensure(gg_data.gpiospec.np);
+
 	gpiochip_find(&gg_data, of_gpiochip_find_and_xlate);
 
 	of_node_put(gg_data.gpiospec.np);
-- 
2.3.6

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC 08/12] drm/tegra: Probe dpaux devices on demand

[Tomeu Vizoso]

When looking up a dpaux device through its DT node, ensure that the
corresponding device has been registered.
---
 drivers/gpu/drm/tegra/dpaux.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/gpu/drm/tegra/dpaux.c b/drivers/gpu/drm/tegra/dpaux.c
index d6b55e3..10bba41 100644
--- a/drivers/gpu/drm/tegra/dpaux.c
+++ b/drivers/gpu/drm/tegra/dpaux.c
@@ -12,6 +12,7 @@
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/of_gpio.h>
+#include <linux/of_platform.h>
 #include <linux/platform_device.h>
 #include <linux/reset.h>
 #include <linux/regulator/consumer.h>
@@ -397,6 +398,8 @@ struct tegra_dpaux *tegra_dpaux_find_by_of_node(struct device_node *np)
 {
 	struct tegra_dpaux *dpaux;
 
+	of_platform_device_ensure(np);
+
 	mutex_lock(&dpaux_lock);
 
 	list_for_each_entry(dpaux, &dpaux_list, list)
-- 
2.3.6

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC 05/12] gpio: Probe pinctrl devices on demand

[Tomeu Vizoso]

Ensure that the device corresponding to the DT node of the pin
controller has been registered.
---
 drivers/gpio/gpiolib-of.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
index 190183d..e26e105 100644
--- a/drivers/gpio/gpiolib-of.c
+++ b/drivers/gpio/gpiolib-of.c
@@ -362,6 +362,8 @@ static void of_gpiochip_add_pin_range(struct gpio_chip *chip)
 		if (ret)
 			break;
 
+		of_platform_device_ensure(pinspec.np);
+
 		pctldev = of_pinctrl_get(pinspec.np);
 		if (!pctldev)
 			break;
-- 
2.3.6

[RFC 06/12] regulator: core: Probe regulators on demand

[Tomeu Vizoso]

When looking up a regulator through its DT node, ensure that the
corresponding device has been registered.
---
 drivers/regulator/core.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index dbc64f7..eee8d45 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -26,6 +26,7 @@
 #include <linux/gpio.h>
 #include <linux/gpio/consumer.h>
 #include <linux/of.h>
+#include <linux/of_platform.h>
 #include <linux/regmap.h>
 #include <linux/regulator/of_regulator.h>
 #include <linux/regulator/consumer.h>
@@ -1286,6 +1287,7 @@ static struct regulator_dev *regulator_dev_lookup(struct device *dev,
 	if (dev && dev->of_node) {
 		node = of_get_regulator(dev, supply);
 		if (node) {
+			of_platform_device_ensure(node);
 			list_for_each_entry(r, &regulator_list, list)
 				if (r->dev.parent &&
 					node == r->dev.of_node)
-- 
2.3.6

[RFC 07/12] drm: Probe panels on demand

[Tomeu Vizoso]

When looking up a panel through its DT node, ensure that the
corresponding device has been registered.
---
 drivers/gpu/drm/drm_panel.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/gpu/drm/drm_panel.c b/drivers/gpu/drm/drm_panel.c
index 2ef988e..041211e 100644
--- a/drivers/gpu/drm/drm_panel.c
+++ b/drivers/gpu/drm/drm_panel.c
@@ -23,6 +23,7 @@
 
 #include <linux/err.h>
 #include <linux/module.h>
+#include <linux/of_platform.h>
 
 #include <drm/drm_crtc.h>
 #include <drm/drm_panel.h>
@@ -80,6 +81,8 @@ struct drm_panel *of_drm_find_panel(struct device_node *np)
 {
 	struct drm_panel *panel;
 
+	of_platform_device_ensure(np);
+
 	mutex_lock(&panel_lock);
 
 	list_for_each_entry(panel, &panel_list, list) {
-- 
2.3.6

[RFC 09/12] i2c: core: Probe i2c master devices on demand

[Tomeu Vizoso]

When looking up an i2c master through its DT node, ensure that the
corresponding device has been registered.
---
 drivers/i2c/i2c-core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/i2c/i2c-core.c b/drivers/i2c/i2c-core.c
index 098f698..53a3588 100644
--- a/drivers/i2c/i2c-core.c
+++ b/drivers/i2c/i2c-core.c
@@ -40,6 +40,7 @@
 #include <linux/of.h>
 #include <linux/of_device.h>
 #include <linux/of_irq.h>
+#include <linux/of_platform.h>
 #include <linux/clk/clk-conf.h>
 #include <linux/completion.h>
 #include <linux/hardirq.h>
@@ -1339,6 +1340,8 @@ struct i2c_adapter *of_find_i2c_adapter_by_node(struct device_node *node)
 {
 	struct device *dev;
 
+	of_platform_device_ensure(node);
+
 	dev = bus_find_device(&i2c_bus_type, NULL, node,
 					 of_dev_node_match);
 	if (!dev)
-- 
2.3.6

[RFC 10/12] pwm: Probe PWM chip devices on demand

[Tomeu Vizoso]

When looking up a PWM chip through its DT node, ensure that the
corresponding device has been registered.
---
 drivers/pwm/core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pwm/core.c b/drivers/pwm/core.c
index ba34c7d..31f486c 100644
--- a/drivers/pwm/core.c
+++ b/drivers/pwm/core.c
@@ -24,6 +24,7 @@
 #include <linux/radix-tree.h>
 #include <linux/list.h>
 #include <linux/mutex.h>
+#include <linux/of_platform.h>
 #include <linux/err.h>
 #include <linux/slab.h>
 #include <linux/device.h>
@@ -479,6 +480,8 @@ static struct pwm_chip *of_node_to_pwmchip(struct device_node *np)
 {
 	struct pwm_chip *chip;
 
+	of_platform_device_ensure(np);
+
 	mutex_lock(&pwm_lock);
 
 	list_for_each_entry(chip, &pwm_chips, list)
-- 
2.3.6

[RFC 11/12] backlight: Probe backlight devices on demand

[Tomeu Vizoso]

When looking up a backlight device through its DT node, ensure that the
corresponding device has been registered.
---
 drivers/video/backlight/backlight.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/video/backlight/backlight.c b/drivers/video/backlight/backlight.c
index bddc8b1..fe7833b 100644
--- a/drivers/video/backlight/backlight.c
+++ b/drivers/video/backlight/backlight.c
@@ -12,6 +12,7 @@
 #include <linux/device.h>
 #include <linux/backlight.h>
 #include <linux/notifier.h>
+#include <linux/of_platform.h>
 #include <linux/ctype.h>
 #include <linux/err.h>
 #include <linux/fb.h>
@@ -559,6 +560,8 @@ struct backlight_device *of_find_backlight_by_node(struct device_node *node)
 {
 	struct device *dev;
 
+	of_platform_device_ensure(node);
+
 	dev = class_find_device(backlight_class, NULL, node, of_parent_match);
 
 	return dev ? to_backlight_device(dev) : NULL;
-- 
2.3.6

[RFC 12/12] usb: phy: Probe phy devices on demand

[Tomeu Vizoso]

When looking up a phy through its DT node, ensure that the corresponding
device has been registered.
---
 drivers/usb/phy/phy.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/usb/phy/phy.c b/drivers/usb/phy/phy.c
index d1cd6b5..7084f21 100644
--- a/drivers/usb/phy/phy.c
+++ b/drivers/usb/phy/phy.c
@@ -13,6 +13,7 @@
 #include <linux/err.h>
 #include <linux/device.h>
 #include <linux/module.h>
+#include <linux/of_platform.h>
 #include <linux/slab.h>
 #include <linux/of.h>
 
@@ -192,6 +193,8 @@ struct usb_phy *devm_usb_get_phy_by_phandle(struct device *dev,
 		goto err0;
 	}
 
+	of_platform_device_ensure(node);
+
 	spin_lock_irqsave(&phy_lock, flags);
 
 	phy = __of_usb_find_phy(node);
-- 
2.3.6

Re: [RFC 00/12] On-demand device registration

[Alexander Holler]

Am 24.04.2015 um 16:47 schrieb Tomeu Vizoso:
> Hi,
> 
> while reading the thread [0] that Alexander Holler started with his series to make probing order deterministic, it occurred to me that it should be possible to achieve the same by probing devices as they are referenced by other devices.
> 
> This basically reuses the information that is already embedded in the probe() implementations, saving us from refactoring existing drivers or adding information to DTBs.
> 
> The main issue I see is that the registration code path in some subsystems may not be reentrant, so some refactoring of the locking will be needed. In my testing I have found this problem with regulators, as the supply of a regulator might end up being registered during the registration of the first one.
> 
> Something I'm not completely happy with is that I have had to move the population of the device tree after all platform drivers have been registered. Otherwise I don't see how I could register drivers on demand as we don't have yet each driver's compatible strings.
> 
> I have done my testing on a Tegra124-based Chromebook, and these patches were enough to eliminate all the deferred probes.

First you have to solve a problem which is totally unrelated to DT or
ACPI or x86 or ARM:

I think as long as drivers don't register themself whithout any side
effect, this problem isn't solvable. In order to make an ordered list of
drivers to start, you need to know which drivers are actually available.

And also drivers are registering themself with their initcall, they
might do an awfull lot of stuff besides just registering themself. That
means several drivers already have prerequisites and dependcies for
their initcall. That means you can't just call their initcall to get and
idea of which driver an initcall is even part of.

That ends up with the fact that you just don't have a list of drivers
you can sort, based on whatever algorithm you might have in mind.

I've tried to solve that problem with marking drivers which don't have
any prerequisits (and side effects) as "well done".

The patch which did that was 5/9 in my series, this one:

https://lkml.org/lkml/2014/5/12/414

Unfortunately nobody seemed really interested and without one of the few
"big guys" in your pocket, it's absolutely impossible to get such
changes into the kernel.

Not to speak about all the unavoidable discussions about absolutely
silly things.

But maybe I'm the problem here. No idea. I wish you more luck than I had
in the past two or three years.

Regards,

Alexander Holler

Re: [RFC 00/12] On-demand device registration

[Tomeu Vizoso]

On 25 April 2015 at 01:15, Alexander Holler <holler@ahsoftware.de> wrote:
> Am 24.04.2015 um 16:47 schrieb Tomeu Vizoso:
>> Hi,
>>
>> while reading the thread [0] that Alexander Holler started with his series to make probing order deterministic, it occurred to me that it should be possible to achieve the same by probing devices as they are referenced by other devices.
>>
>> This basically reuses the information that is already embedded in the probe() implementations, saving us from refactoring existing drivers or adding information to DTBs.
>>
>> The main issue I see is that the registration code path in some subsystems may not be reentrant, so some refactoring of the locking will be needed. In my testing I have found this problem with regulators, as the supply of a regulator might end up being registered during the registration of the first one.
>>
>> Something I'm not completely happy with is that I have had to move the population of the device tree after all platform drivers have been registered. Otherwise I don't see how I could register drivers on demand as we don't have yet each driver's compatible strings.
>>
>> I have done my testing on a Tegra124-based Chromebook, and these patches were enough to eliminate all the deferred probes.
>
> First you have to solve a problem which is totally unrelated to DT or
> ACPI or x86 or ARM:
>
> I think as long as drivers don't register themself whithout any side
> effect, this problem isn't solvable. In order to make an ordered list of
> drivers to start, you need to know which drivers are actually available.

Yeah, I kind of side-stepped that issue by waiting until all drivers
have been registered before registering devices. I think someone
suggested doing so in your thread (maybe Grant?).

There's lots of things that can be improved regarding driver and
device initialization, but we have to start somewhere :)

Thanks,

Tomeu

> And also drivers are registering themself with their initcall, they
> might do an awfull lot of stuff besides just registering themself. That
> means several drivers already have prerequisites and dependcies for
> their initcall. That means you can't just call their initcall to get and
> idea of which driver an initcall is even part of.
>
> That ends up with the fact that you just don't have a list of drivers
> you can sort, based on whatever algorithm you might have in mind.
>
> I've tried to solve that problem with marking drivers which don't have
> any prerequisits (and side effects) as "well done".
>
> The patch which did that was 5/9 in my series, this one:
>
> https://lkml.org/lkml/2014/5/12/414
>
> Unfortunately nobody seemed really interested and without one of the few
> "big guys" in your pocket, it's absolutely impossible to get such
> changes into the kernel.
>
> Not to speak about all the unavoidable discussions about absolutely
> silly things.
>
> But maybe I'm the problem here. No idea. I wish you more luck than I had
> in the past two or three years.
>
> Regards,
>
> Alexander Holler
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC 00/12] On-demand device registration

[Alexander Holler]

Am 28.04.2015 um 14:49 schrieb Tomeu Vizoso:
> On 25 April 2015 at 01:15, Alexander Holler <holler@ahsoftware.de> wrote:
>> Am 24.04.2015 um 16:47 schrieb Tomeu Vizoso:
>>> Hi,
>>>
>>> while reading the thread [0] that Alexander Holler started with his series to make probing order deterministic, it occurred to me that it should be possible to achieve the same by probing devices as they are referenced by other devices.
>>>
>>> This basically reuses the information that is already embedded in the probe() implementations, saving us from refactoring existing drivers or adding information to DTBs.
>>>
>>> The main issue I see is that the registration code path in some subsystems may not be reentrant, so some refactoring of the locking will be needed. In my testing I have found this problem with regulators, as the supply of a regulator might end up being registered during the registration of the first one.
>>>
>>> Something I'm not completely happy with is that I have had to move the population of the device tree after all platform drivers have been registered. Otherwise I don't see how I could register drivers on demand as we don't have yet each driver's compatible strings.
>>>
>>> I have done my testing on a Tegra124-based Chromebook, and these patches were enough to eliminate all the deferred probes.
>>
>> First you have to solve a problem which is totally unrelated to DT or
>> ACPI or x86 or ARM:
>>
>> I think as long as drivers don't register themself whithout any side
>> effect, this problem isn't solvable. In order to make an ordered list of
>> drivers to start, you need to know which drivers are actually available.
>
> Yeah, I kind of side-stepped that issue by waiting until all drivers
> have been registered before registering devices. I think someone
> suggested doing so in your thread (maybe Grant?).

That doesn't work. As said above, several drivers doing a lot more than 
just registering in their initcall. They might even crash if some 
prerequisits aren't given. And several of these prerequisits (init 
orders) are hardcoded by various means.

> There's lots of things that can be improved regarding driver and
> device initialization, but we have to start somewhere :)

That's what I've tried by marking good drivers as such (and by placing 
them in their own initcall level group).

But as said, good luck. I've tried it already and nobody seemed 
interested. Some people even seem to panic if they see a patch which 
changes something in linux/init/ ;)

Regards,

Alexander Holler

>
> Thanks,
>
> Tomeu
>
>> And also drivers are registering themself with their initcall, they
>> might do an awfull lot of stuff besides just registering themself. That
>> means several drivers already have prerequisites and dependcies for
>> their initcall. That means you can't just call their initcall to get and
>> idea of which driver an initcall is even part of.
>>
>> That ends up with the fact that you just don't have a list of drivers
>> you can sort, based on whatever algorithm you might have in mind.
>>
>> I've tried to solve that problem with marking drivers which don't have
>> any prerequisits (and side effects) as "well done".
>>
>> The patch which did that was 5/9 in my series, this one:
>>
>> https://lkml.org/lkml/2014/5/12/414
>>
>> Unfortunately nobody seemed really interested and without one of the few
>> "big guys" in your pocket, it's absolutely impossible to get such
>> changes into the kernel.
>>
>> Not to speak about all the unavoidable discussions about absolutely
>> silly things.
>>
>> But maybe I'm the problem here. No idea. I wish you more luck than I had
>> in the past two or three years.
>>
>> Regards,
>>
>> Alexander Holler
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC 00/12] On-demand device registration

[Tomeu Vizoso]

On 28 April 2015 at 20:17, Alexander Holler <holler-SXC+2es9fhnfWeYVQQPykw@public.gmane.org> wrote:
> Am 28.04.2015 um 14:49 schrieb Tomeu Vizoso:
>>
>> On 25 April 2015 at 01:15, Alexander Holler <holler-SXC+2es9fhnfWeYVQQPykw@public.gmane.org> wrote:
>>>
>>> Am 24.04.2015 um 16:47 schrieb Tomeu Vizoso:
>>>>
>>>> Hi,
>>>>
>>>> while reading the thread [0] that Alexander Holler started with his
>>>> series to make probing order deterministic, it occurred to me that it should
>>>> be possible to achieve the same by probing devices as they are referenced by
>>>> other devices.
>>>>
>>>> This basically reuses the information that is already embedded in the
>>>> probe() implementations, saving us from refactoring existing drivers or
>>>> adding information to DTBs.
>>>>
>>>> The main issue I see is that the registration code path in some
>>>> subsystems may not be reentrant, so some refactoring of the locking will be
>>>> needed. In my testing I have found this problem with regulators, as the
>>>> supply of a regulator might end up being registered during the registration
>>>> of the first one.
>>>>
>>>> Something I'm not completely happy with is that I have had to move the
>>>> population of the device tree after all platform drivers have been
>>>> registered. Otherwise I don't see how I could register drivers on demand as
>>>> we don't have yet each driver's compatible strings.
>>>>
>>>> I have done my testing on a Tegra124-based Chromebook, and these patches
>>>> were enough to eliminate all the deferred probes.
>>>
>>>
>>> First you have to solve a problem which is totally unrelated to DT or
>>> ACPI or x86 or ARM:
>>>
>>> I think as long as drivers don't register themself whithout any side
>>> effect, this problem isn't solvable. In order to make an ordered list of
>>> drivers to start, you need to know which drivers are actually available.
>>
>>
>> Yeah, I kind of side-stepped that issue by waiting until all drivers
>> have been registered before registering devices. I think someone
>> suggested doing so in your thread (maybe Grant?).
>
>
> That doesn't work. As said above, several drivers doing a lot more than just
> registering in their initcall. They might even crash if some prerequisits
> aren't given. And several of these prerequisits (init orders) are hardcoded
> by various means.

But aren't those dependencies being taken care currently by the
initcall level the driver is placed in? That remains the same in this
approach.

I have obviously spent less time thinking about all this than you, so
sorry if I'm missing too many points.

Regards,

Tomeu

>> There's lots of things that can be improved regarding driver and
>> device initialization, but we have to start somewhere :)
>
>
> That's what I've tried by marking good drivers as such (and by placing them
> in their own initcall level group).
>
> But as said, good luck. I've tried it already and nobody seemed interested.
> Some people even seem to panic if they see a patch which changes something
> in linux/init/ ;)
>
> Regards,
>
> Alexander Holler
>
>
>>
>> Thanks,
>>
>> Tomeu
>>
>>> And also drivers are registering themself with their initcall, they
>>> might do an awfull lot of stuff besides just registering themself. That
>>> means several drivers already have prerequisites and dependcies for
>>> their initcall. That means you can't just call their initcall to get and
>>> idea of which driver an initcall is even part of.
>>>
>>> That ends up with the fact that you just don't have a list of drivers
>>> you can sort, based on whatever algorithm you might have in mind.
>>>
>>> I've tried to solve that problem with marking drivers which don't have
>>> any prerequisits (and side effects) as "well done".
>>>
>>> The patch which did that was 5/9 in my series, this one:
>>>
>>> https://lkml.org/lkml/2014/5/12/414
>>>
>>> Unfortunately nobody seemed really interested and without one of the few
>>> "big guys" in your pocket, it's absolutely impossible to get such
>>> changes into the kernel.
>>>
>>> Not to speak about all the unavoidable discussions about absolutely
>>> silly things.
>>>
>>> But maybe I'm the problem here. No idea. I wish you more luck than I had
>>> in the past two or three years.
>>>
>>> Regards,
>>>
>>> Alexander Holler
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
>>> in
>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC 00/12] On-demand device registration

[Alexander Holler]

Am 29.04.2015 um 08:58 schrieb Tomeu Vizoso:
> On 28 April 2015 at 20:17, Alexander Holler <holler@ahsoftware.de> wrote:
>> Am 28.04.2015 um 14:49 schrieb Tomeu Vizoso:
>>>
>>> On 25 April 2015 at 01:15, Alexander Holler <holler@ahsoftware.de> wrote:
>>>>
>>>> Am 24.04.2015 um 16:47 schrieb Tomeu Vizoso:
>>>>>
>>>>> Hi,
>>>>>
>>>>> while reading the thread [0] that Alexander Holler started with his
>>>>> series to make probing order deterministic, it occurred to me that it should
>>>>> be possible to achieve the same by probing devices as they are referenced by
>>>>> other devices.
>>>>>
>>>>> This basically reuses the information that is already embedded in the
>>>>> probe() implementations, saving us from refactoring existing drivers or
>>>>> adding information to DTBs.
>>>>>
>>>>> The main issue I see is that the registration code path in some
>>>>> subsystems may not be reentrant, so some refactoring of the locking will be
>>>>> needed. In my testing I have found this problem with regulators, as the
>>>>> supply of a regulator might end up being registered during the registration
>>>>> of the first one.
>>>>>
>>>>> Something I'm not completely happy with is that I have had to move the
>>>>> population of the device tree after all platform drivers have been
>>>>> registered. Otherwise I don't see how I could register drivers on demand as
>>>>> we don't have yet each driver's compatible strings.
>>>>>
>>>>> I have done my testing on a Tegra124-based Chromebook, and these patches
>>>>> were enough to eliminate all the deferred probes.
>>>>
>>>>
>>>> First you have to solve a problem which is totally unrelated to DT or
>>>> ACPI or x86 or ARM:
>>>>
>>>> I think as long as drivers don't register themself whithout any side
>>>> effect, this problem isn't solvable. In order to make an ordered list of
>>>> drivers to start, you need to know which drivers are actually available.
>>>
>>>
>>> Yeah, I kind of side-stepped that issue by waiting until all drivers
>>> have been registered before registering devices. I think someone
>>> suggested doing so in your thread (maybe Grant?).
>>
>>
>> That doesn't work. As said above, several drivers doing a lot more than just
>> registering in their initcall. They might even crash if some prerequisits
>> aren't given. And several of these prerequisits (init orders) are hardcoded
>> by various means.
>
> But aren't those dependencies being taken care currently by the
> initcall level the driver is placed in? That remains the same in this
> approach.

In short, no. There are various very ugly things done in several drivers 
to enforce some order.

Regards,

Alexander Holler

Re: [RFC 00/12] On-demand device registration

[Alexander Holler]

Am 29.04.2015 um 11:46 schrieb Alexander Holler:
> Am 29.04.2015 um 08:58 schrieb Tomeu Vizoso:
>> On 28 April 2015 at 20:17, Alexander Holler <holler@ahsoftware.de> wrote:
>>> Am 28.04.2015 um 14:49 schrieb Tomeu Vizoso:
>>>>
>>>> On 25 April 2015 at 01:15, Alexander Holler <holler@ahsoftware.de>
>>>> wrote:
>>>>>
>>>>> Am 24.04.2015 um 16:47 schrieb Tomeu Vizoso:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> while reading the thread [0] that Alexander Holler started with his
>>>>>> series to make probing order deterministic, it occurred to me that
>>>>>> it should
>>>>>> be possible to achieve the same by probing devices as they are
>>>>>> referenced by
>>>>>> other devices.
>>>>>>
>>>>>> This basically reuses the information that is already embedded in the
>>>>>> probe() implementations, saving us from refactoring existing
>>>>>> drivers or
>>>>>> adding information to DTBs.
>>>>>>
>>>>>> The main issue I see is that the registration code path in some
>>>>>> subsystems may not be reentrant, so some refactoring of the
>>>>>> locking will be
>>>>>> needed. In my testing I have found this problem with regulators,
>>>>>> as the
>>>>>> supply of a regulator might end up being registered during the
>>>>>> registration
>>>>>> of the first one.
>>>>>>
>>>>>> Something I'm not completely happy with is that I have had to move
>>>>>> the
>>>>>> population of the device tree after all platform drivers have been
>>>>>> registered. Otherwise I don't see how I could register drivers on
>>>>>> demand as
>>>>>> we don't have yet each driver's compatible strings.
>>>>>>
>>>>>> I have done my testing on a Tegra124-based Chromebook, and these
>>>>>> patches
>>>>>> were enough to eliminate all the deferred probes.
>>>>>
>>>>>
>>>>> First you have to solve a problem which is totally unrelated to DT or
>>>>> ACPI or x86 or ARM:
>>>>>
>>>>> I think as long as drivers don't register themself whithout any side
>>>>> effect, this problem isn't solvable. In order to make an ordered
>>>>> list of
>>>>> drivers to start, you need to know which drivers are actually
>>>>> available.
>>>>
>>>>
>>>> Yeah, I kind of side-stepped that issue by waiting until all drivers
>>>> have been registered before registering devices. I think someone
>>>> suggested doing so in your thread (maybe Grant?).
>>>
>>>
>>> That doesn't work. As said above, several drivers doing a lot more
>>> than just
>>> registering in their initcall. They might even crash if some
>>> prerequisits
>>> aren't given. And several of these prerequisits (init orders) are
>>> hardcoded
>>> by various means.
>>
>> But aren't those dependencies being taken care currently by the
>> initcall level the driver is placed in? That remains the same in this
>> approach.
>
> In short, no. There are various very ugly things done in several drivers
> to enforce some order.

To explain it a bit more:

The case with the regulator you've described above is just one of these 
ugly things done on some driver initcalls. That's why I've decided to 
introduce a new initcall level which only contains drivers which don't 
have any side effect but just registering themself. Ideally all drivers 
would end up in that level, and thus the special level would not be 
necessary at all. Because this means that several drivers have to 
change, which is a lot of work, an intermediate workaround is to make a 
whitelist. That's easier than a blacklist which would mean you have to 
examine every driver. And that whitelist is exactly what I did by 
introducing those "well done" drivers.

And just in case of, I haven't looked at your patches at all. I've just 
read the introductory mail and the subjects of the patches and then 
concluded what I've written.

That means if you've a special use case in mind, your solution might 
work. But as an overall solution the problem with identifying drivers 
(mapping initcalls to drivers) has to be solved (which I've tried with 
those "well done" driver list).

Regards,

Alexander Holler