[Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Ayush Singh]

Hello everyone,


This mailing thread is dedicated to discussing what approach should be 
taken regarding overlays for addon board + connector setups. It is 
loosely a continuation of [0], so feel free to go through it.


Introduction
*************

To provide a background, the goal is to have a common way to support 
setups involving addon board + connector. Some examples are as follows:

- MikroBUS [1]

- PocketBeagle 2 [2]: After discussion in [0], it seems that even board 
headers should be treated as a addon board + connector problem.


There are 2 main approaches currently floating around. They serve as 
examples of Global and Local approaches. However, the main discussion 
topic is global vs local rather than the specific approaches.


1. __symbols__ based approach [3]


This was originally proposed by Andre Davis [3]. It defines an overlay 
with just special names in `__symbols__`, which is used along with an 
overlay for the addon-board, which makes use of the node names defined 
in the connector `__symbols__` overlay. Please take a look at the 
original patch series since it provides a working example of how it can 
be used [3].


It has a few nice benefits such as it works pretty well with existing 
infrastructure, and does not need much in the way of new functionality. 
However, for this discussion thread, I want to consolidate the 
discussion regarding how this approach directly adds the devices to the 
appropriate nodes, Eg. An SPI device in addon board will be added to the 
appropriate SPI controller, etc. This means the changes are made to the 
global tree.


2. export-symbols based approach [4]


This approach was originally proposed by Herve Codina [5]. It defines a 
special node (local to the connector) say `export-symbols`. This node 
takes precedence over global `__symbols__`, and thus is used to define 
standard names for nodes used in addon board overlay. Please look at [4] 
to get a more in-depth explanation.


The main difference here is that all the addon board overlay changes are 
isolated to the connector node in the devicetree. Eg: an I2C device in 
addon board will be added to the connector node (as defined in i2c bus 
extension patch series [6]). This means the changes are made to the 
connector node and not the global tree.


This approach needs extra plumbing (like i2c bus extension [6], 
something similar for SPI, etc) to make the whole approach work. Only 
GPIO and PWM with Nexus node can use this approach right now.


Basic Requirements

*********************


Here are some basic functionality that the chosen approach can do for it 
to be viable for the connector + addon board setups:


1. Dynamic device addition/removal from userspace


A lot of connectors + addon board setups do not have any dynamic 
discovery addition. This is compounded when talking about treating the 
whole header in SBCs like PocketBeagle 2 as a connector, since people 
would want to wire LEDs and stuff to individual pins. So a mechanism 
using sysfs or configfs is required


2. Dynamic device addition/removal by driver using EEPROM or something else


Some setups (MikroBUS boards with 1-wire EEPROM, Beagle capes) have 
EEPROMs that contain board information which can be used to detect which 
overlay should be applied.


Main Discussion

*****************

The main topic I wish to discuss if global devicetree overlays are okay 
for addon-board setups. Let me outline some reasons for I prefer the 
local devicetree overlays approach:


1. Addon board removal on multiple connector setups


Say connector A added an I2C device to the controller, then connector B 
added an I2C device to the same controller. I am not sure how well 
removing overlays out-of-order works.


2. Who owns the device


Say there are 2 connectors A and B. Both connectors share an I2C 
controller. Let both connectors have the same device attached. In case 
of `__symbols__` based approach, both connectors would technically be 
successful in applying the overlays, rather than one of the overlays 
failing.


3. How to register the newly added devices


I am a bit unsure about this one since I will have to check if the 
kernel tries to register new devices automatically after applying the 
overlay. For local setups, I was using `devm_of_platform_populate` on 
the connector device.


4. Security


I think local modification is more secure than global tree modification. 
A completely local solution should be as secure from devicetree 
perspective as USB. But I am not an expert.


Drawbacks of local setups

***************************


1. Needs a lot of surrounding work.


I2C bus extension is needed for I2C devices to work, something similar 
for SPI. At least ADC, PWM and GPIO should be covered with just nexus nodes.


Closing Thoughts

******************


I would really like to reach consensus regarding weather the addon-board 
overlays should be global or local. This will help to give a direction 
regarding what should be improved, and hopefully make future development 
move faster. Once a bit of consensus has been reached, we can discuss 
specific implementations.


[0]: 
https://lore.kernel.org/linux-devicetree/d42100cb-eaa0-487f-aaaa-6d8f87bc0705@beagleboard.org/T/#m09b2ebe28b6202b2a926970150caf718eff6d9ac

[1]: https://www.mikroe.com/mikrobus

[2]: https://www.beagleboard.org/boards/pocketbeagle-2

[3]: https://lore.kernel.org/lkml/20240702164403.29067-1-afd@ti.com/

[4]: 
https://lore.kernel.org/devicetree-spec/dbe566ea-447f-4f91-a0b2-f464374955f4@beagleboard.org/T/#m591e737b48ebe96aafa39d87652e07eef99dff90

[5]: 
https://lore.kernel.org/all/20241209151830.95723-1-herve.codina@bootlin.com/

[6]: 
https://lore.kernel.org/devicetree-spec/20250401081041.114333-1-herve.codina@bootlin.com/T/#t



Best Regards,

Ayush Singh

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Herve Codina]

Hi Ayush,

Thanks for this discussion initiative!


On Wed, 30 Apr 2025 17:37:33 +0530
Ayush Singh <ayush@beagleboard.org> wrote:

...

> 1. __symbols__ based approach [3]
> 
> 
> This was originally proposed by Andre Davis [3]. It defines an overlay 
> with just special names in `__symbols__`, which is used along with an 
> overlay for the addon-board, which makes use of the node names defined 
> in the connector `__symbols__` overlay. Please take a look at the 
> original patch series since it provides a working example of how it can 
> be used [3].
> 

The __symbols__ based approach needs 2 overlays to handle the case where
2 connectors (A and B) are present an you want to connect a board described
by a single overlay.

The first overlay applied "adapts" the __symbols__ node for the connector
where the board is connected (for instance connector A) in order to have
the symbols used by the overlay describing the board resolved to the
correct symbols.

I think this open race conditions when the overlay is applied by the kernel
itself. Indeed, we need to perform 2 steps in an atomic way:
  1) Adapt symbols
  2) Applied board overlay

Also, a 3rd step should be added to restore symbols modified to their
original value after the overlay is applied. This should be done to avoid
any symbols collision.

An other negative point is that properties in __symbols__ node are not
described by device-tree bindings.
How can we ensure interoperability between different base board.

The export-symbols node is a node fully described in the DT and it is
a sub-node of a connector node. This connector node has compatible string
and a binding describing its own property and sub-nodes. Among them, the
export-symbols node is described and can be checks by dt_binding_check.

This implies that whatever the base board, for a given connector type
(compatible string) the properties inside the export-symbols node have the
exact same name. Any addon board overlay designed for this connector type
will apply whatever the system where this connector is soldered.

> 
> It has a few nice benefits such as it works pretty well with existing 
> infrastructure, and does not need much in the way of new functionality. 

With existing infrastructure in the kernel, it leads to memory leaks if you
add or change a property in an existing node.

In other word, each time you update a symbol in the __symbols__ node, you
have a memory leak.

> However, for this discussion thread, I want to consolidate the 
> discussion regarding how this approach directly adds the devices to the 
> appropriate nodes, Eg. An SPI device in addon board will be added to the 
> appropriate SPI controller, etc. This means the changes are made to the 
> global tree.
> 
> 

...

> 
> Basic Requirements
> 
> *********************
> 
> 
> Here are some basic functionality that the chosen approach can do for it 
> to be viable for the connector + addon board setups:
> 
> 
> 1. Dynamic device addition/removal from userspace
> 
> 
> A lot of connectors + addon board setups do not have any dynamic 
> discovery addition. This is compounded when talking about treating the 
> whole header in SBCs like PocketBeagle 2 as a connector, since people 
> would want to wire LEDs and stuff to individual pins. So a mechanism 
> using sysfs or configfs is required

request_firmware() or the firmware upload feature (CONFIG_FW_UPLOAD) could
also be used if the connector is seen as a specific device and has a driver.
  https://elixir.bootlin.com/linux/v6.15-rc3/source/Documentation/driver-api/firmware/fw_upload.rst

> 
> 
> 2. Dynamic device addition/removal by driver using EEPROM or something else
> 
> 
> Some setups (MikroBUS boards with 1-wire EEPROM, Beagle capes) have 
> EEPROMs that contain board information which can be used to detect which 
> overlay should be applied.
> 
> 
> Main Discussion
> 
> *****************
> 
> The main topic I wish to discuss if global devicetree overlays are okay 
> for addon-board setups. Let me outline some reasons for I prefer the 
> local devicetree overlays approach:
> 
> 
> 1. Addon board removal on multiple connector setups
> 
> 
> Say connector A added an I2C device to the controller, then connector B 
> added an I2C device to the same controller. I am not sure how well 
> removing overlays out-of-order works.
> 
> 
> 2. Who owns the device
> 
> 
> Say there are 2 connectors A and B. Both connectors share an I2C 
> controller. Let both connectors have the same device attached. In case 
> of `__symbols__` based approach, both connectors would technically be 
> successful in applying the overlays, rather than one of the overlays 
> failing.
> 
> 
> 3. How to register the newly added devices
> 
> 
> I am a bit unsure about this one since I will have to check if the 
> kernel tries to register new devices automatically after applying the 
> overlay. For local setups, I was using `devm_of_platform_populate` on 
> the connector device.

It depends on the bus the device is added.
when an overlay is applied by the kernel, OF_RECONFIG_* events are
triggered. Some buses handle them:

	$ git grep OF_RECONFIG_CHANGE
	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
	drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
	drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
	drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
	include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
	include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,

> 
> 
> 4. Security
> 
> 
> I think local modification is more secure than global tree modification. 
> A completely local solution should be as secure from devicetree 
> perspective as USB. But I am not an expert.
> 
> 
> Drawbacks of local setups
> 
> ***************************
> 
> 
> 1. Needs a lot of surrounding work.
> 
> 
> I2C bus extension is needed for I2C devices to work, something similar 
> for SPI. At least ADC, PWM and GPIO should be covered with just nexus nodes.

I wouldn't say 'a lot'.
I already did the work for I2C bus extension [0] and the implementation was
not so complex.

[0] https://lore.kernel.org/all/20250401081041.114333-1-herve.codina@bootlin.com/

> 
> 
> Closing Thoughts
> 
> ******************
> 
> 
> I would really like to reach consensus regarding weather the addon-board 
> overlays should be global or local. This will help to give a direction 
> regarding what should be improved, and hopefully make future development 
> move faster. Once a bit of consensus has been reached, we can discuss 
> specific implementations.
> 


Again, thanks again for initiating this discussion.
Hope this will help to move forward on this topic!

Best regards,
Hervé

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Geert Uytterhoeven]

Hi Hervé,

On Wed, 30 Apr 2025 at 16:09, Herve Codina <herve.codina@bootlin.com> wrote:
> On Wed, 30 Apr 2025 17:37:33 +0530
> Ayush Singh <ayush@beagleboard.org> wrote:
>
> ...
>
> > 1. __symbols__ based approach [3]
> >
> >
> > This was originally proposed by Andre Davis [3]. It defines an overlay
> > with just special names in `__symbols__`, which is used along with an
> > overlay for the addon-board, which makes use of the node names defined
> > in the connector `__symbols__` overlay. Please take a look at the
> > original patch series since it provides a working example of how it can
> > be used [3].
> >
>
> The __symbols__ based approach needs 2 overlays to handle the case where
> 2 connectors (A and B) are present an you want to connect a board described
> by a single overlay.
>
> The first overlay applied "adapts" the __symbols__ node for the connector
> where the board is connected (for instance connector A) in order to have
> the symbols used by the overlay describing the board resolved to the
> correct symbols.
>
> I think this open race conditions when the overlay is applied by the kernel
> itself. Indeed, we need to perform 2 steps in an atomic way:
>   1) Adapt symbols
>   2) Applied board overlay

I think that can be solved by not applying them in two steps, but by
in-memory merging of the symbol and board overlays first, and applying
the result.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Herve Codina]

Hi Geert,

On Wed, 30 Apr 2025 16:31:17 +0200
Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> Hi Hervé,
> 
> On Wed, 30 Apr 2025 at 16:09, Herve Codina <herve.codina@bootlin.com> wrote:
> > On Wed, 30 Apr 2025 17:37:33 +0530
> > Ayush Singh <ayush@beagleboard.org> wrote:
> >
> > ...
> >  
> > > 1. __symbols__ based approach [3]
> > >
> > >
> > > This was originally proposed by Andre Davis [3]. It defines an overlay
> > > with just special names in `__symbols__`, which is used along with an
> > > overlay for the addon-board, which makes use of the node names defined
> > > in the connector `__symbols__` overlay. Please take a look at the
> > > original patch series since it provides a working example of how it can
> > > be used [3].
> > >  
> >
> > The __symbols__ based approach needs 2 overlays to handle the case where
> > 2 connectors (A and B) are present an you want to connect a board described
> > by a single overlay.
> >
> > The first overlay applied "adapts" the __symbols__ node for the connector
> > where the board is connected (for instance connector A) in order to have
> > the symbols used by the overlay describing the board resolved to the
> > correct symbols.
> >
> > I think this open race conditions when the overlay is applied by the kernel
> > itself. Indeed, we need to perform 2 steps in an atomic way:
> >   1) Adapt symbols
> >   2) Applied board overlay  
> 
> I think that can be solved by not applying them in two steps, but by
> in-memory merging of the symbol and board overlays first, and applying
> the result.

Yes, indeed.

This implies a significant work for the __symbols__ based approach. I think
we can say that "it works pretty well with existing infrastructure" is no
more fully true.

Best regards,
Hervé

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Ayush Singh]

On 4/30/25 19:39, Herve Codina wrote:

> Hi Ayush,
>
> Thanks for this discussion initiative!
>
>
> On Wed, 30 Apr 2025 17:37:33 +0530
> Ayush Singh <ayush@beagleboard.org> wrote:
>
> ...
>
>> 1. __symbols__ based approach [3]
>>
>>
>> This was originally proposed by Andre Davis [3]. It defines an overlay
>> with just special names in `__symbols__`, which is used along with an
>> overlay for the addon-board, which makes use of the node names defined
>> in the connector `__symbols__` overlay. Please take a look at the
>> original patch series since it provides a working example of how it can
>> be used [3].
>>
> The __symbols__ based approach needs 2 overlays to handle the case where
> 2 connectors (A and B) are present an you want to connect a board described
> by a single overlay.
>
> The first overlay applied "adapts" the __symbols__ node for the connector
> where the board is connected (for instance connector A) in order to have
> the symbols used by the overlay describing the board resolved to the
> correct symbols.
>
> I think this open race conditions when the overlay is applied by the kernel
> itself. Indeed, we need to perform 2 steps in an atomic way:
>    1) Adapt symbols
>    2) Applied board overlay
>
> Also, a 3rd step should be added to restore symbols modified to their
> original value after the overlay is applied. This should be done to avoid
> any symbols collision.
>
> An other negative point is that properties in __symbols__ node are not
> described by device-tree bindings.
> How can we ensure interoperability between different base board.
>
> The export-symbols node is a node fully described in the DT and it is
> a sub-node of a connector node. This connector node has compatible string
> and a binding describing its own property and sub-nodes. Among them, the
> export-symbols node is described and can be checks by dt_binding_check.
>
> This implies that whatever the base board, for a given connector type
> (compatible string) the properties inside the export-symbols node have the
> exact same name. Any addon board overlay designed for this connector type
> will apply whatever the system where this connector is soldered.
>
>> It has a few nice benefits such as it works pretty well with existing
>> infrastructure, and does not need much in the way of new functionality.
> With existing infrastructure in the kernel, it leads to memory leaks if you
> add or change a property in an existing node.
>
> In other word, each time you update a symbol in the __symbols__ node, you
> have a memory leak.
>
>> However, for this discussion thread, I want to consolidate the
>> discussion regarding how this approach directly adds the devices to the
>> appropriate nodes, Eg. An SPI device in addon board will be added to the
>> appropriate SPI controller, etc. This means the changes are made to the
>> global tree.
>>
>>
> ...
>
>> Basic Requirements
>>
>> *********************
>>
>>
>> Here are some basic functionality that the chosen approach can do for it
>> to be viable for the connector + addon board setups:
>>
>>
>> 1. Dynamic device addition/removal from userspace
>>
>>
>> A lot of connectors + addon board setups do not have any dynamic
>> discovery addition. This is compounded when talking about treating the
>> whole header in SBCs like PocketBeagle 2 as a connector, since people
>> would want to wire LEDs and stuff to individual pins. So a mechanism
>> using sysfs or configfs is required
> request_firmware() or the firmware upload feature (CONFIG_FW_UPLOAD) could
> also be used if the connector is seen as a specific device and has a driver.
>    https://elixir.bootlin.com/linux/v6.15-rc3/source/Documentation/driver-api/firmware/fw_upload.rst

Yup, my current plan is to use request_firmware. If it is good enough 
security wise for remoteproc, it should be fine here as well.


>
>>
>> 2. Dynamic device addition/removal by driver using EEPROM or something else
>>
>>
>> Some setups (MikroBUS boards with 1-wire EEPROM, Beagle capes) have
>> EEPROMs that contain board information which can be used to detect which
>> overlay should be applied.
>>
>>
>> Main Discussion
>>
>> *****************
>>
>> The main topic I wish to discuss if global devicetree overlays are okay
>> for addon-board setups. Let me outline some reasons for I prefer the
>> local devicetree overlays approach:
>>
>>
>> 1. Addon board removal on multiple connector setups
>>
>>
>> Say connector A added an I2C device to the controller, then connector B
>> added an I2C device to the same controller. I am not sure how well
>> removing overlays out-of-order works.
>>
>>
>> 2. Who owns the device
>>
>>
>> Say there are 2 connectors A and B. Both connectors share an I2C
>> controller. Let both connectors have the same device attached. In case
>> of `__symbols__` based approach, both connectors would technically be
>> successful in applying the overlays, rather than one of the overlays
>> failing.
>>
>>
>> 3. How to register the newly added devices
>>
>>
>> I am a bit unsure about this one since I will have to check if the
>> kernel tries to register new devices automatically after applying the
>> overlay. For local setups, I was using `devm_of_platform_populate` on
>> the connector device.
> It depends on the bus the device is added.
> when an overlay is applied by the kernel, OF_RECONFIG_* events are
> triggered. Some buses handle them:
>
> 	$ git grep OF_RECONFIG_CHANGE
> 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
> 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
> 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
> 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
> 	drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
> 	drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
> 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
> 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
> 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
> 	include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
> 	include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,
>
>>
>> 4. Security
>>
>>
>> I think local modification is more secure than global tree modification.
>> A completely local solution should be as secure from devicetree
>> perspective as USB. But I am not an expert.
>>
>>
>> Drawbacks of local setups
>>
>> ***************************
>>
>>
>> 1. Needs a lot of surrounding work.
>>
>>
>> I2C bus extension is needed for I2C devices to work, something similar
>> for SPI. At least ADC, PWM and GPIO should be covered with just nexus nodes.
> I wouldn't say 'a lot'.
> I already did the work for I2C bus extension [0] and the implementation was
> not so complex.
>
> [0] https://lore.kernel.org/all/20250401081041.114333-1-herve.codina@bootlin.com/

Well, when I say a lot of work, I don't mean codewise. That will be 
pretty simple actually. The problem I foresee is convincing everything 
in different subsystems. Basically, a lot of discussions.

Still, I do think figuring these things out now would be best in the 
long run. After all, while I cannot say for everyone else, I want to use 
this setup for at least MikroBUS, Grove, PocketBeagle 2 connector, 
BeagleBone Black connector, and RPI (for BeagleY-AI) connector.

>>
>> Closing Thoughts
>>
>> ******************
>>
>>
>> I would really like to reach consensus regarding weather the addon-board
>> overlays should be global or local. This will help to give a direction
>> regarding what should be improved, and hopefully make future development
>> move faster. Once a bit of consensus has been reached, we can discuss
>> specific implementations.
>>
>
> Again, thanks again for initiating this discussion.
> Hope this will help to move forward on this topic!
>
> Best regards,
> Hervé


Ayush Singh

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Ayush Singh]

> It depends on the bus the device is added.
> when an overlay is applied by the kernel, OF_RECONFIG_* events are
> triggered. Some buses handle them:
>
> 	$ git grep OF_RECONFIG_CHANGE
> 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
> 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
> 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
> 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
> 	drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
> 	drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
> 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
> 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
> 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
> 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
> 	include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
> 	include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,


Well, if some bus does handle the event, I guess it is a bug in the 
subsystems that do not? Maybe Greg Kroah-Hartman can clarify the 
expected behavior here? Maybe we are in transition phase here.


Also I guess, feel free to suggest anyone else I should add to this 
discussion thread. I want everyone to come to an understanding regarding 
the fundamental behavior. I have been getting mixed signals in the 
various discussions I have had till now.


Ayush Singh

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Greg Kroah-Hartman]

On Sun, May 04, 2025 at 06:03:26PM +0530, Ayush Singh wrote:
> > It depends on the bus the device is added.
> > when an overlay is applied by the kernel, OF_RECONFIG_* events are
> > triggered. Some buses handle them:
> > 
> > 	$ git grep OF_RECONFIG_CHANGE
> > 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
> > 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
> > 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > 	drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
> > 	drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
> > 	drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
> > 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
> > 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
> > 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
> > 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > 	include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
> > 	include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,
> 
> 
> Well, if some bus does handle the event, I guess it is a bug in the
> subsystems that do not? Maybe Greg Kroah-Hartman can clarify the expected
> behavior here? Maybe we are in transition phase here.

Perhaps those other busses just do not have OF devices and so they never
needed to add that functionality to them?

If they do, then by all means add that code.  OF devices are not
possible for many bus types, so there shouldn't be a need to add this to
the driver core, right?

thanks,

greg k-h

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Ayush Singh]

On 5/4/25 18:20, Greg Kroah-Hartman wrote:
> On Sun, May 04, 2025 at 06:03:26PM +0530, Ayush Singh wrote:
>>> It depends on the bus the device is added.
>>> when an overlay is applied by the kernel, OF_RECONFIG_* events are
>>> triggered. Some buses handle them:
>>>
>>> 	$ git grep OF_RECONFIG_CHANGE
>>> 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
>>> 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
>>> 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
>>> 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
>>> 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
>>> 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
>>> 	drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
>>> 	drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
>>> 	drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
>>> 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
>>> 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
>>> 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
>>> 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
>>> 	include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
>>> 	include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,
>>
>> Well, if some bus does handle the event, I guess it is a bug in the
>> subsystems that do not? Maybe Greg Kroah-Hartman can clarify the expected
>> behavior here? Maybe we are in transition phase here.
> Perhaps those other busses just do not have OF devices and so they never
> needed to add that functionality to them?
>
> If they do, then by all means add that code.  OF devices are not
> possible for many bus types, so there shouldn't be a need to add this to
> the driver core, right?
>
> thanks,
>
> greg k-h


UART devices are pretty common in both Beagle capes and MikroBUS. So I 
think that will probably need to be added at some point.


Ayush Singh

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Greg Kroah-Hartman]

On Sun, May 04, 2025 at 06:30:24PM +0530, Ayush Singh wrote:
> 
> On 5/4/25 18:20, Greg Kroah-Hartman wrote:
> > On Sun, May 04, 2025 at 06:03:26PM +0530, Ayush Singh wrote:
> > > > It depends on the bus the device is added.
> > > > when an overlay is applied by the kernel, OF_RECONFIG_* events are
> > > > triggered. Some buses handle them:
> > > > 
> > > > 	$ git grep OF_RECONFIG_CHANGE
> > > > 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
> > > > 	drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
> > > > 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > 	drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > 	drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > 	drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
> > > > 	drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
> > > > 	drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
> > > > 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
> > > > 	drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
> > > > 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > 	drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > 	include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
> > > > 	include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,
> > > 
> > > Well, if some bus does handle the event, I guess it is a bug in the
> > > subsystems that do not? Maybe Greg Kroah-Hartman can clarify the expected
> > > behavior here? Maybe we are in transition phase here.
> > Perhaps those other busses just do not have OF devices and so they never
> > needed to add that functionality to them?
> > 
> > If they do, then by all means add that code.  OF devices are not
> > possible for many bus types, so there shouldn't be a need to add this to
> > the driver core, right?
> > 
> > thanks,
> > 
> > greg k-h
> 
> 
> UART devices are pretty common in both Beagle capes and MikroBUS. So I think
> that will probably need to be added at some point.

uarts are not a bus, they are a type of device that is implemented by
many different bus drivers (pci, USB, etc.)

thanks,

greg k-h

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Geert Uytterhoeven]

On Sun, 4 May 2025 at 15:13, Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
> On Sun, May 04, 2025 at 06:30:24PM +0530, Ayush Singh wrote:
> > On 5/4/25 18:20, Greg Kroah-Hartman wrote:
> > > On Sun, May 04, 2025 at 06:03:26PM +0530, Ayush Singh wrote:
> > > > > It depends on the bus the device is added.
> > > > > when an overlay is applied by the kernel, OF_RECONFIG_* events are
> > > > > triggered. Some buses handle them:
> > > > >
> > > > >         $ git grep OF_RECONFIG_CHANGE
> > > > >         drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
> > > > >         drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
> > > > >         drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > >         drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > >         drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > >         drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > >         drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
> > > > >         drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
> > > > >         drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
> > > > >         drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
> > > > >         drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
> > > > >         drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > >         drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > >         include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
> > > > >         include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,
> > > >
> > > > Well, if some bus does handle the event, I guess it is a bug in the
> > > > subsystems that do not? Maybe Greg Kroah-Hartman can clarify the expected

Support for OF_RECONFIG_* events was added only to buses where users
had a need for it (spi, i2c, platform, weim) and to the GPIO subsystem
(for hogs).  More can be added...

> > > > behavior here? Maybe we are in transition phase here.
> > > Perhaps those other busses just do not have OF devices and so they never
> > > needed to add that functionality to them?
> > >
> > > If they do, then by all means add that code.  OF devices are not
> > > possible for many bus types, so there shouldn't be a need to add this to
> > > the driver core, right?
> >
> > UART devices are pretty common in both Beagle capes and MikroBUS. So I think
> > that will probably need to be added at some point.
>
> uarts are not a bus, they are a type of device that is implemented by
> many different bus drivers (pci, USB, etc.)

It depends...
https://elixir.bootlin.com/linux/v6.14.5/source/Documentation/devicetree/bindings/serial/serial.yaml#L90

It makes perfect sense to add support for OF_RECONFIG_* events to the
serial bus, so people can attach e.g. a bluetooth or GNSS device to a
serial header on their board (or even to a legacy serial port on a PC),
load a DT overlay, and enjoy a working device.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Rob Herring]

On Mon, May 5, 2025 at 1:44 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> On Sun, 4 May 2025 at 15:13, Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
> > On Sun, May 04, 2025 at 06:30:24PM +0530, Ayush Singh wrote:
> > > On 5/4/25 18:20, Greg Kroah-Hartman wrote:
> > > > On Sun, May 04, 2025 at 06:03:26PM +0530, Ayush Singh wrote:
> > > > > > It depends on the bus the device is added.
> > > > > > when an overlay is applied by the kernel, OF_RECONFIG_* events are
> > > > > > triggered. Some buses handle them:
> > > > > >
> > > > > >         $ git grep OF_RECONFIG_CHANGE
> > > > > >         drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_ADD:
> > > > > >         drivers/bus/imx-weim.c: case OF_RECONFIG_CHANGE_REMOVE:
> > > > > >         drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > > >         drivers/gpio/gpiolib-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > > >         drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > > >         drivers/i2c/i2c-core-of.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > > >         drivers/of/dynamic.c: * Return: OF_RECONFIG_CHANGE_REMOVE on device going from enabled to
> > > > > >         drivers/of/dynamic.c: * disabled, OF_RECONFIG_CHANGE_ADD on device going from disabled to
> > > > > >         drivers/of/dynamic.c:   return new_state ? OF_RECONFIG_CHANGE_ADD : OF_RECONFIG_CHANGE_REMOVE;
> > > > > >         drivers/of/platform.c:  case OF_RECONFIG_CHANGE_ADD:
> > > > > >         drivers/of/platform.c:  case OF_RECONFIG_CHANGE_REMOVE:
> > > > > >         drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_ADD:
> > > > > >         drivers/spi/spi.c:      case OF_RECONFIG_CHANGE_REMOVE:
> > > > > >         include/linux/of.h:     OF_RECONFIG_CHANGE_ADD,
> > > > > >         include/linux/of.h:     OF_RECONFIG_CHANGE_REMOVE,
> > > > >
> > > > > Well, if some bus does handle the event, I guess it is a bug in the
> > > > > subsystems that do not? Maybe Greg Kroah-Hartman can clarify the expected
>
> Support for OF_RECONFIG_* events was added only to buses where users
> had a need for it (spi, i2c, platform, weim) and to the GPIO subsystem
> (for hogs).  More can be added...
>
> > > > > behavior here? Maybe we are in transition phase here.
> > > > Perhaps those other busses just do not have OF devices and so they never
> > > > needed to add that functionality to them?
> > > >
> > > > If they do, then by all means add that code.  OF devices are not
> > > > possible for many bus types, so there shouldn't be a need to add this to
> > > > the driver core, right?
> > >
> > > UART devices are pretty common in both Beagle capes and MikroBUS. So I think
> > > that will probably need to be added at some point.
> >
> > uarts are not a bus, they are a type of device that is implemented by
> > many different bus drivers (pci, USB, etc.)
>
> It depends...
> https://elixir.bootlin.com/linux/v6.14.5/source/Documentation/devicetree/bindings/serial/serial.yaml#L90
>
> It makes perfect sense to add support for OF_RECONFIG_* events to the
> serial bus, so people can attach e.g. a bluetooth or GNSS device to a
> serial header on their board (or even to a legacy serial port on a PC),
> load a DT overlay, and enjoy a working device.

serdev has bigger issues than lack of OF_RECONFIG_* support. Whether a
UART is bound to tty or serdev is purely based on whether there's a
child node or not under the UART node. Until someone figures out how
to make unbinding from tty layer work or allow both at the same time,
then it's not going to matter.

Notifier events are usually a solution waiting for something better. I
think we should actually remove the property notifiers. While we can
support nodes coming and going, we absolutely have no support for
properties coming and going. I don't think we want to start
refcounting properties either.

Rob

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Andrew Davis]

On 4/30/25 7:07 AM, Ayush Singh wrote:
> Hello everyone,
> 
> 
> This mailing thread is dedicated to discussing what approach should be taken regarding overlays for addon board + connector setups. It is loosely a continuation of [0], so feel free to go through it.
> 
> 
> Introduction
> *************
> 
> To provide a background, the goal is to have a common way to support setups involving addon board + connector. Some examples are as follows:
> 
> - MikroBUS [1]
> 
> - PocketBeagle 2 [2]: After discussion in [0], it seems that even board headers should be treated as a addon board + connector problem.
> 
> 
> There are 2 main approaches currently floating around. They serve as examples of Global and Local approaches. However, the main discussion topic is global vs local rather than the specific approaches.
> 

This is far too reductive, local vs global is only a minor detail as part
of a couple proposed solutions to a much larger topic. There are much more
fundamental issues around add-on boards to be worked out before we get to
this level of detail.

> 
> 1. __symbols__ based approach [3]
> 
> 
> This was originally proposed by Andre Davis [3]. It defines an overlay with just special names in `__symbols__`, which is used along with an overlay for the addon-board, which makes use of the node names defined in the connector `__symbols__` overlay. Please take a look at the original patch series since it provides a working example of how it can be used [3].
> 
> 
> It has a few nice benefits such as it works pretty well with existing infrastructure, and does not need much in the way of new functionality. However, for this discussion thread, I want to consolidate the discussion regarding how this approach directly adds the devices to the appropriate nodes, Eg. An SPI device in addon board will be added to the appropriate SPI controller, etc. This means the changes are made to the global tree.
> 

This is how devices are already added, if you add a SPI device you
put it under the SPI controller. There is no "global tree", there
is just one device tree to represent the device, and it has a well
established ways of hierarchically representing devices.

> 
> 2. export-symbols based approach [4]
> 
> 
> This approach was originally proposed by Herve Codina [5]. It defines a special node (local to the connector) say `export-symbols`. This node takes precedence over global `__symbols__`, and thus is used to define standard names for nodes used in addon board overlay. Please look at [4] to get a more in-depth explanation.
> 
> 
> The main difference here is that all the addon board overlay changes are isolated to the connector node in the devicetree. Eg: an I2C device in addon board will be added to the connector node (as defined in i2c bus extension patch series [6]). This means the changes are made to the connector node and not the global tree.
> 
> 
> This approach needs extra plumbing (like i2c bus extension [6], something similar for SPI, etc) to make the whole approach work. Only GPIO and PWM with Nexus node can use this approach right now.
> 

This is the more fundamental issue: do you want to work within the existing
device tree framework, or do you want to make something new. `export-symbols`
is only the tip of the iceberg of new things that will be needed.

This "extra plumbing" is bus extensions in DT for every wired bus type.
Countless new Nexus nodes, connector definitions, pinmuxing, and bindings
for the same. And then changes to DT overlay tooling including all the
infrastructure that uses overlays in many external projects, which IMHO is
the largest issue here.

My core point here is this, imagine you take a BeagleBone and plug in
a cape. This statically combined device in your hand can be completely
described using existing DT today. For any solution we come up with for
doing this dynamically, the resulting DT should look the same as if
it was done for the combined device by hand ahead of time. The combined
Beagle+cape device is exactly the same device no matter what time you
plug in the cape. So why should the DT that describes this combined
device be different in the two cases?

> 
> Basic Requirements
> 
> *********************
> 
> 
> Here are some basic functionality that the chosen approach can do for it to be viable for the connector + addon board setups:
> 
> 
> 1. Dynamic device addition/removal from userspace
> 

I'm going to suggest we ignore the removal part. Not because it is too
difficult to solve, but because it is impossible to solve.

A huge amount of drivers and devices do not actually allow for removal.

The reason is because there is no need, hot-pluggable busses are the
exception, not the rule. The rare cases like USB are built to handle
this both in hardware and software. None of the connectors we have
talked about are actually hot-pluggable! I2C, SPI, etc.. none of
these are hot-pluggable. Even if you get away with yanking the cape
off a BeagleBone while it is running once or twice, it is violating
the electrical specifications and you will eventually break something.

If we don't focus on the (non-valid) removal part, so many other parts
solutions become viable again. Right now we have no good way to even
*add* an add-on board, even statically, so let's not let "perfect" be
the enemy of good..

> 
> A lot of connectors + addon board setups do not have any dynamic discovery addition. This is compounded when talking about treating the whole header in SBCs like PocketBeagle 2 as a connector, since people would want to wire LEDs and stuff to individual pins. So a mechanism using sysfs or configfs is required
> 
> 
> 2. Dynamic device addition/removal by driver using EEPROM or something else
> 
> 
> Some setups (MikroBUS boards with 1-wire EEPROM, Beagle capes) have EEPROMs that contain board information which can be used to detect which overlay should be applied.
> 
> 
> Main Discussion
> 
> *****************
> 
> The main topic I wish to discuss if global devicetree overlays are okay for addon-board setups. Let me outline some reasons for I prefer the local devicetree overlays approach:
> 
> 
> 1. Addon board removal on multiple connector setups
> 
> 
> Say connector A added an I2C device to the controller, then connector B added an I2C device to the same controller. I am not sure how well removing overlays out-of-order works.
> 

I don't follow here. Multiple of the same I2C device are still identifiable
even on the same controller. Their IDs give them away, and even if not, this
would be trivially solved with a little bit of meta-data tracking in the
driver model layer.

> 
> 2. Who owns the device
> 
> 
> Say there are 2 connectors A and B. Both connectors share an I2C controller. Let both connectors have the same device attached. In case of `__symbols__` based approach, both connectors would technically be successful in applying the overlays, rather than one of the overlays failing.
> 

How does the connector solution help here? Seems the `__symbols__` way would make
it easier to detect as both devices would need added to the same controller node,
which would be the point to detect such conflicts. Were each connector would have
no way to communicate that the resource is double-booked.

> 
> 3. How to register the newly added devices
> 
> 
> I am a bit unsure about this one since I will have to check if the kernel tries to register new devices automatically after applying the overlay. For local setups, I was using `devm_of_platform_populate` on the connector device.
> 

Easy to solve for either method, we can always re-run the device registration loop
after applying an overlay if it doesn't already.

> 
> 4. Security
> 
> 
> I think local modification is more secure than global tree modification. A completely local solution should be as secure from devicetree perspective as USB. But I am not an expert.
> 
> 
> Drawbacks of local setups
> 
> ***************************
> 
> 
> 1. Needs a lot of surrounding work.
> 
> 
> I2C bus extension is needed for I2C devices to work, something similar for SPI. At least ADC, PWM and GPIO should be covered with just nexus nodes.
> 
> 
> Closing Thoughts
> 
> ******************
> 
> 
> I would really like to reach consensus regarding weather the addon-board overlays should be global or local. This will help to give a direction regarding what should be improved, and hopefully make future development move faster. Once a bit of consensus has been reached, we can discuss specific implementations.
> 


There is one big benefit of evil vendor trees: you can test things out like
this before sending things upstream. You are working with the Beagle folks,
and that has to be the most perfect testing ground for add-on board overlay
solutions.

So I would suggest you take the solution you like the most and implement it
in the Beagle ecosystem. Try it out and find the pros and cons first hand,
see if it can even work in the first place.

Asking everyone to all reach a consensus on the "right" solution by
talking it out is not going to happen here. This has been an open issue
for way too long for it to be that easy :)

Andrew

> 
> [0]: https://lore.kernel.org/linux-devicetree/d42100cb-eaa0-487f-aaaa-6d8f87bc0705@beagleboard.org/T/#m09b2ebe28b6202b2a926970150caf718eff6d9ac
> 
> [1]: https://www.mikroe.com/mikrobus
> 
> [2]: https://www.beagleboard.org/boards/pocketbeagle-2
> 
> [3]: https://lore.kernel.org/lkml/20240702164403.29067-1-afd@ti.com/
> 
> [4]: https://lore.kernel.org/devicetree-spec/dbe566ea-447f-4f91-a0b2-f464374955f4@beagleboard.org/T/#m591e737b48ebe96aafa39d87652e07eef99dff90
> 
> [5]: https://lore.kernel.org/all/20241209151830.95723-1-herve.codina@bootlin.com/
> 
> [6]: https://lore.kernel.org/devicetree-spec/20250401081041.114333-1-herve.codina@bootlin.com/T/#t
> 
> 
> 
> Best Regards,
> 
> Ayush Singh
> 

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Ayush Singh]

On 6/5/25 00:33, Andrew Davis wrote:

> On 4/30/25 7:07 AM, Ayush Singh wrote:
>> Hello everyone,
>>
>>
>> This mailing thread is dedicated to discussing what approach should 
>> be taken regarding overlays for addon board + connector setups. It is 
>> loosely a continuation of [0], so feel free to go through it.
>>
>>
>> Introduction
>> *************
>>
>> To provide a background, the goal is to have a common way to support 
>> setups involving addon board + connector. Some examples are as follows:
>>
>> - MikroBUS [1]
>>
>> - PocketBeagle 2 [2]: After discussion in [0], it seems that even 
>> board headers should be treated as a addon board + connector problem.
>>
>>
>> There are 2 main approaches currently floating around. They serve as 
>> examples of Global and Local approaches. However, the main discussion 
>> topic is global vs local rather than the specific approaches.
>>
>
> This is far too reductive, local vs global is only a minor detail as part
> of a couple proposed solutions to a much larger topic. There are much 
> more
> fundamental issues around add-on boards to be worked out before we get to
> this level of detail.


Well, I am not sure that is quite true. The problem is that it seems to 
be a major blocker for things like sysfs and configfs devicetree overlay 
patches. And if we go with the global route, well, we are pretty much 
re-implementing those patches. Maybe we can add some way to check that 
only i2c-connecotr, spi-connector etc parent nodes for a add-board 
connector are modified, but well, not sure if that is enough.


>
>>
>> 1. __symbols__ based approach [3]
>>
>>
>> This was originally proposed by Andre Davis [3]. It defines an 
>> overlay with just special names in `__symbols__`, which is used along 
>> with an overlay for the addon-board, which makes use of the node 
>> names defined in the connector `__symbols__` overlay. Please take a 
>> look at the original patch series since it provides a working example 
>> of how it can be used [3].
>>
>>
>> It has a few nice benefits such as it works pretty well with existing 
>> infrastructure, and does not need much in the way of new 
>> functionality. However, for this discussion thread, I want to 
>> consolidate the discussion regarding how this approach directly adds 
>> the devices to the appropriate nodes, Eg. An SPI device in addon 
>> board will be added to the appropriate SPI controller, etc. This 
>> means the changes are made to the global tree.
>>
>
> This is how devices are already added, if you add a SPI device you
> put it under the SPI controller. There is no "global tree", there
> is just one device tree to represent the device, and it has a well
> established ways of hierarchically representing devices.
>
>>
>> 2. export-symbols based approach [4]
>>
>>
>> This approach was originally proposed by Herve Codina [5]. It defines 
>> a special node (local to the connector) say `export-symbols`. This 
>> node takes precedence over global `__symbols__`, and thus is used to 
>> define standard names for nodes used in addon board overlay. Please 
>> look at [4] to get a more in-depth explanation.
>>
>>
>> The main difference here is that all the addon board overlay changes 
>> are isolated to the connector node in the devicetree. Eg: an I2C 
>> device in addon board will be added to the connector node (as defined 
>> in i2c bus extension patch series [6]). This means the changes are 
>> made to the connector node and not the global tree.
>>
>>
>> This approach needs extra plumbing (like i2c bus extension [6], 
>> something similar for SPI, etc) to make the whole approach work. Only 
>> GPIO and PWM with Nexus node can use this approach right now.
>>
>
> This is the more fundamental issue: do you want to work within the 
> existing
> device tree framework, or do you want to make something new. 
> `export-symbols`
> is only the tip of the iceberg of new things that will be needed.
>
> This "extra plumbing" is bus extensions in DT for every wired bus type.
> Countless new Nexus nodes, connector definitions, pinmuxing, and bindings
> for the same. And then changes to DT overlay tooling including all the
> infrastructure that uses overlays in many external projects, which 
> IMHO is
> the largest issue here.
>
> My core point here is this, imagine you take a BeagleBone and plug in
> a cape. This statically combined device in your hand can be completely
> described using existing DT today. For any solution we come up with for
> doing this dynamically, the resulting DT should look the same as if
> it was done for the combined device by hand ahead of time. The combined
> Beagle+cape device is exactly the same device no matter what time you
> plug in the cape. So why should the DT that describes this combined
> device be different in the two cases?
>
>>
>> Basic Requirements
>>
>> *********************
>>
>>
>> Here are some basic functionality that the chosen approach can do for 
>> it to be viable for the connector + addon board setups:
>>
>>
>> 1. Dynamic device addition/removal from userspace
>>
>
> I'm going to suggest we ignore the removal part. Not because it is too
> difficult to solve, but because it is impossible to solve.
>
> A huge amount of drivers and devices do not actually allow for removal.
>
> The reason is because there is no need, hot-pluggable busses are the
> exception, not the rule. The rare cases like USB are built to handle
> this both in hardware and software. None of the connectors we have
> talked about are actually hot-pluggable! I2C, SPI, etc.. none of
> these are hot-pluggable. Even if you get away with yanking the cape
> off a BeagleBone while it is running once or twice, it is violating
> the electrical specifications and you will eventually break something.
>
> If we don't focus on the (non-valid) removal part, so many other parts
> solutions become viable again. Right now we have no good way to even
> *add* an add-on board, even statically, so let's not let "perfect" be
> the enemy of good..


Not quite, removal is a very important part of the equation, specially 
for mikroBUS. mikroBUS iteself is not exactly hot-pluggable, but Beagle 
has a usecase of mikroBUS over greybus (over 6lowpan). Since the node 
can be removed, you now have a setup where mikroBUS becomes removable.


>
>>
>> A lot of connectors + addon board setups do not have any dynamic 
>> discovery addition. This is compounded when talking about treating 
>> the whole header in SBCs like PocketBeagle 2 as a connector, since 
>> people would want to wire LEDs and stuff to individual pins. So a 
>> mechanism using sysfs or configfs is required
>>
>>
>> 2. Dynamic device addition/removal by driver using EEPROM or 
>> something else
>>
>>
>> Some setups (MikroBUS boards with 1-wire EEPROM, Beagle capes) have 
>> EEPROMs that contain board information which can be used to detect 
>> which overlay should be applied.
>>
>>
>> Main Discussion
>>
>> *****************
>>
>> The main topic I wish to discuss if global devicetree overlays are 
>> okay for addon-board setups. Let me outline some reasons for I prefer 
>> the local devicetree overlays approach:
>>
>>
>> 1. Addon board removal on multiple connector setups
>>
>>
>> Say connector A added an I2C device to the controller, then connector 
>> B added an I2C device to the same controller. I am not sure how well 
>> removing overlays out-of-order works.
>>
>
> I don't follow here. Multiple of the same I2C device are still 
> identifiable
> even on the same controller. Their IDs give them away, and even if 
> not, this
> would be trivially solved with a little bit of meta-data tracking in the
> driver model layer.


By ids do you mean I2C address or the driver id?

What I was mentioning here is as follows:


add-board:

```

&CONNECTOR_I2C {

     sensor: sensor@12 {

         ...

     };

};

```


We apply the same addon board overlay to connector A and connector B. 
And CONNECTOR_I2C is same for both connectors.


This will resolve to:


```

&CONNECTOR_I2C {

     ...

     sensor: sensor@12 {

         ...

     };

};

```


There will be no errors but the sensor would be registered only once.


In case of a local system, 2 sensor nodes will be present in the global 
devicetree, and one of the sensors will fail during registration.



>
>>
>> 2. Who owns the device
>>
>>
>> Say there are 2 connectors A and B. Both connectors share an I2C 
>> controller. Let both connectors have the same device attached. In 
>> case of `__symbols__` based approach, both connectors would 
>> technically be successful in applying the overlays, rather than one 
>> of the overlays failing.
>>
>
> How does the connector solution help here? Seems the `__symbols__` way 
> would make
> it easier to detect as both devices would need added to the same 
> controller node,
> which would be the point to detect such conflicts. Were each connector 
> would have
> no way to communicate that the resource is double-booked.


You can see the current devicetree at `/sys/firmware/devicetree`, so 
there is that.


Also, when applying devicetree overlay, if the node already exists, the 
properties are over written. The overlay application will not fail. And 
in many cases, the property changes will not even be handled by the 
sensor/device driver.

Also, things like `/delete-node/` and stuff do not work in overlays.


>
>>
>> 3. How to register the newly added devices
>>
>>
>> I am a bit unsure about this one since I will have to check if the 
>> kernel tries to register new devices automatically after applying the 
>> overlay. For local setups, I was using `devm_of_platform_populate` on 
>> the connector device.
>>
>
> Easy to solve for either method, we can always re-run the device 
> registration loop
> after applying an overlay if it doesn't already.
>
>>
>> 4. Security
>>
>>
>> I think local modification is more secure than global tree 
>> modification. A completely local solution should be as secure from 
>> devicetree perspective as USB. But I am not an expert.
>>
>>
>> Drawbacks of local setups
>>
>> ***************************
>>
>>
>> 1. Needs a lot of surrounding work.
>>
>>
>> I2C bus extension is needed for I2C devices to work, something 
>> similar for SPI. At least ADC, PWM and GPIO should be covered with 
>> just nexus nodes.
>>
>>
>> Closing Thoughts
>>
>> ******************
>>
>>
>> I would really like to reach consensus regarding weather the 
>> addon-board overlays should be global or local. This will help to 
>> give a direction regarding what should be improved, and hopefully 
>> make future development move faster. Once a bit of consensus has been 
>> reached, we can discuss specific implementations.
>>
>
>
> There is one big benefit of evil vendor trees: you can test things out 
> like
> this before sending things upstream. You are working with the Beagle 
> folks,
> and that has to be the most perfect testing ground for add-on board 
> overlay
> solutions.
>
> So I would suggest you take the solution you like the most and 
> implement it
> in the Beagle ecosystem. Try it out and find the pros and cons first 
> hand,
> see if it can even work in the first place.
>
> Asking everyone to all reach a consensus on the "right" solution by
> talking it out is not going to happen here. This has been an open issue
> for way too long for it to be that easy :)
>
> Andrew


Well, I wanted to try an upstream first approach, specially since 
everyone else has given up on upstream. The current situation is quite 
messy. mikroBUS in the wild uses modified greybus manifest. In fact, 
there are official guides that show the greybus manifest approach. But, 
those patches were rejected last year, with the suggestion that upstream 
wants devicetree to be used for this purpose. Btw, manifest was already 
in use for 4 years at that point with greybus manifest for 100s of 
addon-boards.


So I was hesitant to have a 2nd out of tree approach for the same. But 
at this point, it does seem more productive to just go the vendor tree 
route for now and try again in a few years.


>
>>
>> [0]: 
>> https://lore.kernel.org/linux-devicetree/d42100cb-eaa0-487f-aaaa-6d8f87bc0705@beagleboard.org/T/#m09b2ebe28b6202b2a926970150caf718eff6d9ac
>>
>> [1]: https://www.mikroe.com/mikrobus
>>
>> [2]: https://www.beagleboard.org/boards/pocketbeagle-2
>>
>> [3]: https://lore.kernel.org/lkml/20240702164403.29067-1-afd@ti.com/
>>
>> [4]: 
>> https://lore.kernel.org/devicetree-spec/dbe566ea-447f-4f91-a0b2-f464374955f4@beagleboard.org/T/#m591e737b48ebe96aafa39d87652e07eef99dff90
>>
>> [5]: 
>> https://lore.kernel.org/all/20241209151830.95723-1-herve.codina@bootlin.com/
>>
>> [6]: 
>> https://lore.kernel.org/devicetree-spec/20250401081041.114333-1-herve.codina@bootlin.com/T/#t
>>
>>
>>
>> Best Regards,
>>
>> Ayush Singh
>>

Best Regards,

Ayush Singh

Re: [Discussion] Global vs Local devicetree overlays for addon board + connector setups

[Luca Ceresoli]

Hello Andrew,

adding a bit more to the discussion.

On Thu, 5 Jun 2025 12:52:19 +0530
Ayush Singh <ayush@beagleboard.org> wrote:

[...]

> >> Basic Requirements
> >>
> >> *********************
> >>
> >>
> >> Here are some basic functionality that the chosen approach can do for 
> >> it to be viable for the connector + addon board setups:
> >>
> >>
> >> 1. Dynamic device addition/removal from userspace
> >>  
> >
> > I'm going to suggest we ignore the removal part. Not because it is too
> > difficult to solve, but because it is impossible to solve.
> >
> > A huge amount of drivers and devices do not actually allow for removal.
> >
> > The reason is because there is no need, hot-pluggable busses are the
> > exception, not the rule. The rare cases like USB are built to handle
> > this both in hardware and software. None of the connectors we have
> > talked about are actually hot-pluggable! I2C, SPI, etc.. none of
> > these are hot-pluggable. Even if you get away with yanking the cape
> > off a BeagleBone while it is running once or twice, it is violating
> > the electrical specifications and you will eventually break something.
> >
> > If we don't focus on the (non-valid) removal part, so many other parts
> > solutions become viable again. Right now we have no good way to even
> > *add* an add-on board, even statically, so let's not let "perfect" be
> > the enemy of good..  
> 
> 
> Not quite, removal is a very important part of the equation, specially 
> for mikroBUS. mikroBUS iteself is not exactly hot-pluggable, but Beagle 
> has a usecase of mikroBUS over greybus (over 6lowpan). Since the node 
> can be removed, you now have a setup where mikroBUS becomes removable.

Removal is indeed valid.

There are use cases for removal, as discussed at LPC2024 [1][2]. And
for the use case Hervé and I am working on, mentioned at LPC, addition
without removal is totally useless.

Busses that are non-natively hotplug are surely tricky to handle but
they work if 1) the hardware is designed to be electrically safe against
removal (HDMI DDC is a simple example based on I²C) and 2) the drivers
and related subsystems are resilient to removal. We are working on 2,
and I'm working particularly on DRM (some series I sent recently: [3],
[4]).

[1] https://lpc.events/event/18/contributions/1696/attachments/1404/3464/ceresoli-codina-hotplug-overlays.pdf
[2] https://www.youtube.com/watch?v=ACDCiC1r8Xw
[3] https://lore.kernel.org/lkml/20250528-drm-bridge-convert-to-alloc-api-v4-1-f04e698c9a77@bootlin.com/
[4] https://lore.kernel.org/lkml/20250606-drm-bridge-alloc-doc-test-v9-0-b5bf7b43ed92@bootlin.com/

Luca

-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com