devicetree.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [Question] Status of user-space dynamic overlays API
@ 2025-02-22 20:13 Ayush Singh
  2025-02-22 20:31 ` Heinrich Schuchardt
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: Ayush Singh @ 2025-02-22 20:13 UTC (permalink / raw)
  To: xypron.glpk, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Geert Uytterhoeven, Greg Kroah-Hartman,
	David Gibson, Luca Ceresoli, Pantelis Antoniou
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

Hello everyone.

I have been looking at ways to do runtime devicetree overlay 
application, and was just wondering what the current status of the 
different proposals [0], [1] were. They seem to be quite old and I think 
they were already rejected, but due to all the broken links, I am not 
really sure about the exact reasons. Also, maybe we now have the 
solutions to some the blockers at the time.


Let me fist go over some of the use cases where I think dynamic 
devicetree overlays can be useful. I am mostly interested in their use 
in single board computers like PocketBeagle 2 [2], Raspberry Pi [3], etc.


# Uses

## Dynamic Pin muxing

A lot of SBC's aimed for creating hardware projects expose headers, 
where each pin can be used for multiple things like GPIO, I2C, PWM, etc, 
depending on the pinmux. I think Raspberry Pi has it's own solution to 
do userspace pinmux, but if userspace devicetree application was a 
thing, it could probably be used for this. Additionally, being able to 
use dynamic devicetree overlays for pin muxing would allow much easier 
transition to use proper device trees during production.


## Dynamic Sensors/Devices

Using devices such as sensors, external ADCs, EEPROMs, etc are also a 
common usecase in SBC's. A lot of current solutions seem to be designed 
around using user-space drivers in such cases. This is a bit of a shame 
since Linux kernel already has drivers for a lot of these drivers, and 
they are probably going to be of higher quality than most user space 
drivers.


# Challenges

## Security

The concerns regarding security seemed to show up in the other 
proposals. There was a proposal to have a devicetree property to 
allow/deny the application of overlays in some nodes, with default being 
deny. Was it insufficient?


## Memory Leaks

Currently, updating/removing properties leaks memory. Was it one of the 
reasons for the rejection of previous proposals?


Maybe kernel already has some solutions more suited to my usecase that I 
am unware of?


[0]: 
https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t

[1]: https://lore.kernel.org/all/20161220190455.25115-1-xypron.glpk@gmx.de/

[2]: https://www.beagleboard.org/boards/pocketbeagle-2

[3]: https://www.raspberrypi.com/


Best Regards,

Ayush Singh


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-22 20:13 [Question] Status of user-space dynamic overlays API Ayush Singh
@ 2025-02-22 20:31 ` Heinrich Schuchardt
  2025-02-24  5:58   ` Ayush Singh
  2025-02-24  8:37 ` Geert Uytterhoeven
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 12+ messages in thread
From: Heinrich Schuchardt @ 2025-02-22 20:31 UTC (permalink / raw)
  To: Ayush Singh, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Geert Uytterhoeven, Greg Kroah-Hartman,
	David Gibson, Luca Ceresoli, Pantelis Antoniou
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

Am 22. Februar 2025 21:13:56 MEZ schrieb Ayush Singh <ayush@beagleboard.org>:
>Hello everyone.
>
>I have been looking at ways to do runtime devicetree overlay application, and was just wondering what the current status of the different proposals [0], [1] were. They seem to be quite old and I think they were already rejected, but due to all the broken links, I am not really sure about the exact reasons. Also, maybe we now have the solutions to some the blockers at the time.
>
>
>Let me fist go over some of the use cases where I think dynamic devicetree overlays can be useful. I am mostly interested in their use in single board computers like PocketBeagle 2 [2], Raspberry Pi [3], etc.
>
>
># Uses
>
>## Dynamic Pin muxing
>
>A lot of SBC's aimed for creating hardware projects expose headers, where each pin can be used for multiple things like GPIO, I2C, PWM, etc, depending on the pinmux. I think Raspberry Pi has it's own solution to do userspace pinmux, but if userspace devicetree application was a thing, it could probably be used for this. Additionally, being able to use dynamic devicetree overlays for pin muxing would allow much easier transition to use proper device trees during production.
>
>
>## Dynamic Sensors/Devices
>
>Using devices such as sensors, external ADCs, EEPROMs, etc are also a common usecase in SBC's. A lot of current solutions seem to be designed around using user-space drivers in such cases. This is a bit of a shame since Linux kernel already has drivers for a lot of these drivers, and they are probably going to be of higher quality than most user space drivers.
>
>
># Challenges
>
>## Security
>
>The concerns regarding security seemed to show up in the other proposals. There was a proposal to have a devicetree property to allow/deny the application of overlays in some nodes, with default being deny. Was it insufficient?
>
>
>## Memory Leaks
>
>Currently, updating/removing properties leaks memory. Was it one of the reasons for the rejection of previous proposals?
>
>
>Maybe kernel already has some solutions more suited to my usecase that I am unware of?
>
>
>[0]: https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t
>
>[1]: https://lore.kernel.org/all/20161220190455.25115-1-xypron.glpk@gmx.de/
>
>[2]: https://www.beagleboard.org/boards/pocketbeagle-2
>
>[3]: https://www.raspberrypi.com/
>
>
>Best Regards,
>
>Ayush Singh
>

Hello Ayush,

On [1] I gave up when I got the impression that the maintainers only wanted to further their own companies interest and did not show openness for a globally usable functionality.

Best regards

Heinrich


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-22 20:31 ` Heinrich Schuchardt
@ 2025-02-24  5:58   ` Ayush Singh
  0 siblings, 0 replies; 12+ messages in thread
From: Ayush Singh @ 2025-02-24  5:58 UTC (permalink / raw)
  To: Heinrich Schuchardt, Jason Kridner, Deepak Khatri, d-gole,
	Robert Nelson, Andrew Davis, Geert Uytterhoeven,
	Greg Kroah-Hartman, David Gibson, Luca Ceresoli,
	Pantelis Antoniou
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

On 2/23/25 02:01, Heinrich Schuchardt wrote:

> Am 22. Februar 2025 21:13:56 MEZ schrieb Ayush Singh <ayush@beagleboard.org>:
>> Hello everyone.
>>
>> I have been looking at ways to do runtime devicetree overlay application, and was just wondering what the current status of the different proposals [0], [1] were. They seem to be quite old and I think they were already rejected, but due to all the broken links, I am not really sure about the exact reasons. Also, maybe we now have the solutions to some the blockers at the time.
>>
>>
>> Let me fist go over some of the use cases where I think dynamic devicetree overlays can be useful. I am mostly interested in their use in single board computers like PocketBeagle 2 [2], Raspberry Pi [3], etc.
>>
>>
>> # Uses
>>
>> ## Dynamic Pin muxing
>>
>> A lot of SBC's aimed for creating hardware projects expose headers, where each pin can be used for multiple things like GPIO, I2C, PWM, etc, depending on the pinmux. I think Raspberry Pi has it's own solution to do userspace pinmux, but if userspace devicetree application was a thing, it could probably be used for this. Additionally, being able to use dynamic devicetree overlays for pin muxing would allow much easier transition to use proper device trees during production.
>>
>>
>> ## Dynamic Sensors/Devices
>>
>> Using devices such as sensors, external ADCs, EEPROMs, etc are also a common usecase in SBC's. A lot of current solutions seem to be designed around using user-space drivers in such cases. This is a bit of a shame since Linux kernel already has drivers for a lot of these drivers, and they are probably going to be of higher quality than most user space drivers.
>>
>>
>> # Challenges
>>
>> ## Security
>>
>> The concerns regarding security seemed to show up in the other proposals. There was a proposal to have a devicetree property to allow/deny the application of overlays in some nodes, with default being deny. Was it insufficient?
>>
>>
>> ## Memory Leaks
>>
>> Currently, updating/removing properties leaks memory. Was it one of the reasons for the rejection of previous proposals?
>>
>>
>> Maybe kernel already has some solutions more suited to my usecase that I am unware of?
>>
>>
>> [0]: https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t
>>
>> [1]: https://lore.kernel.org/all/20161220190455.25115-1-xypron.glpk@gmx.de/
>>
>> [2]: https://www.beagleboard.org/boards/pocketbeagle-2
>>
>> [3]: https://www.raspberrypi.com/
>>
>>
>> Best Regards,
>>
>> Ayush Singh
>>
> Hello Ayush,
>
> On [1] I gave up when I got the impression that the maintainers only wanted to further their own companies interest and did not show openness for a globally usable functionality.
>
> Best regards
>
> Heinrich
>

I am thinking about an implementation more in line with your sysfs 
proposal [0]. Maybe I should send an RFC to get more replies but here is 
what I was thinking of:


Having a property in `aliases` such as `export_node{num}` to explicitly 
export nodes which support overlay application. The `of_sysfs` module 
will create sysfs entries for each of exported nodes.


The same files as [0] will be provided for each node:

load:   This is a write only file. Data written to it is interpreted as 
devicetree blob, and applied using `of_overlay_fdt_apply()` to the 
target node.

loaded: This is a read only file.
         It provides the count of loaded overlays as a decimal
         number.

unload: This is a write only file.
         If a positive number n is wrtten to this file the n
         most recent overlays are destroyed.
         If a negative number is written to this file all
         overlays are destroyed.


Since any nodes that can have dynamic overlays applied need to be 
explicitly enabled in base devicetree, I think this should help with 
some security concerns.

I am a bit unsure if I would need to create a new device for each of the 
exported node or just create kobject for each node directly (since the 
sysfs entries should probably be under 
`/sys/firmware/overlays/export_node{num}` rather than `/sys/devices`).


[0]: https://lore.kernel.org/all/20161220190455.25115-1-xypron.glpk@gmx.de/


Best Regards,

Ayush Singh


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-22 20:13 [Question] Status of user-space dynamic overlays API Ayush Singh
  2025-02-22 20:31 ` Heinrich Schuchardt
@ 2025-02-24  8:37 ` Geert Uytterhoeven
  2025-02-24 10:09   ` Ayush Singh
  2025-03-05  4:27   ` David Gibson
  2025-02-24 11:24 ` Luca Ceresoli
  2025-04-30 10:18 ` Ayush Singh
  3 siblings, 2 replies; 12+ messages in thread
From: Geert Uytterhoeven @ 2025-02-24  8:37 UTC (permalink / raw)
  To: Ayush Singh
  Cc: xypron.glpk, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Greg Kroah-Hartman, David Gibson, Luca Ceresoli,
	Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

Hi Ayush,

On Sat, 22 Feb 2025 at 21:14, Ayush Singh <ayush@beagleboard.org> wrote:
> # Challenges
>
> ## Security
>
> The concerns regarding security seemed to show up in the other
> proposals. There was a proposal to have a devicetree property to
> allow/deny the application of overlays in some nodes, with default being
> deny. Was it insufficient?

This is the most important issue: using DT overlays, you can change
about anything.  There is no protection yet to limit this to e.g. the
expansion connectors on your board.
This is what the various WIP "connector" abstractions are trying
to solve.

> ## Memory Leaks
>
> Currently, updating/removing properties leaks memory. Was it one of the
> reasons for the rejection of previous proposals?

IMO this is a minor issue. I am sure this can be improved upon.  We just
need some way to keep track of which properties are part of the initial
FDT (and thus can't be freed), and which were allocated dynamically.

> [0]:
> https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t

FTR, I do keep this up to date:
https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-drivers.git/log/?h=topic/overlays

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-24  8:37 ` Geert Uytterhoeven
@ 2025-02-24 10:09   ` Ayush Singh
  2025-02-24 13:29     ` Herve Codina
  2025-03-05  4:31     ` David Gibson
  2025-03-05  4:27   ` David Gibson
  1 sibling, 2 replies; 12+ messages in thread
From: Ayush Singh @ 2025-02-24 10:09 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: xypron.glpk, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Greg Kroah-Hartman, David Gibson, Luca Ceresoli,
	Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

On 2/24/25 14:07, Geert Uytterhoeven wrote:

> Hi Ayush,
>
> On Sat, 22 Feb 2025 at 21:14, Ayush Singh <ayush@beagleboard.org> wrote:
>> # Challenges
>>
>> ## Security
>>
>> The concerns regarding security seemed to show up in the other
>> proposals. There was a proposal to have a devicetree property to
>> allow/deny the application of overlays in some nodes, with default being
>> deny. Was it insufficient?
> This is the most important issue: using DT overlays, you can change
> about anything.  There is no protection yet to limit this to e.g. the
> expansion connectors on your board.
> This is what the various WIP "connector" abstractions are trying
> to solve.

Thanks for clarifying. However, as I mentioned above, there are usecases 
for dynamic overlays outside of connectors. Specifically, for the 
usecase of connecting random sensors to board pins. I do agree that any 
fairly well specified connector should probably have it's own drivers 
rather than using a generic userspace API.

Would it be enough to use `aliases` to specify the nodes where an 
overlay can be applied as I have described here [0]. To further lock 
down things, it might be possible to allow introducing indirection or 
scoping nodes of sort. Here is an example:

\ {

aliases {

export_node0 = &spi0_scoped;

};

};

&spi0 {

spi0_scoped {};

// Stuff already connected internally

};

Any children of `spi0_scoped` should be treated as devices directly 
connected to `spi0` but should provided isolation for any changes that 
userspace overlays can make.

>> ## Memory Leaks
>>
>> Currently, updating/removing properties leaks memory. Was it one of the
>> reasons for the rejection of previous proposals?
> IMO this is a minor issue. I am sure this can be improved upon.  We just
> need some way to keep track of which properties are part of the initial
> FDT (and thus can't be freed), and which were allocated dynamically.
>
>> [0]:
>> https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t
> FTR, I do keep this up to date:
> https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-drivers.git/log/?h=topic/overlays

Thanks, will check the patches.

Is there a list or mailing thread documenting what would be needed to 
move this effort forward. I would love to help in any way possible.

> Gr{oetje,eeting}s,
>
>                          Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
>                                  -- Linus Torvalds


[0]: 
https://lore.kernel.org/all/d5bed265-1dbd-44d1-8287-8ca993624b79@beagleboard.org/


Best Regards,

Ayush Singh


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-22 20:13 [Question] Status of user-space dynamic overlays API Ayush Singh
  2025-02-22 20:31 ` Heinrich Schuchardt
  2025-02-24  8:37 ` Geert Uytterhoeven
@ 2025-02-24 11:24 ` Luca Ceresoli
  2025-04-30 10:18 ` Ayush Singh
  3 siblings, 0 replies; 12+ messages in thread
From: Luca Ceresoli @ 2025-02-24 11:24 UTC (permalink / raw)
  To: Ayush Singh
  Cc: xypron.glpk, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Geert Uytterhoeven, Greg Kroah-Hartman,
	David Gibson, Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	Hervé Codina

Hello,

+Cc: Hervé

Luca

-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-24 10:09   ` Ayush Singh
@ 2025-02-24 13:29     ` Herve Codina
  2025-03-05  4:31     ` David Gibson
  1 sibling, 0 replies; 12+ messages in thread
From: Herve Codina @ 2025-02-24 13:29 UTC (permalink / raw)
  To: Ayush Singh
  Cc: Geert Uytterhoeven, xypron.glpk, Jason Kridner, Deepak Khatri,
	d-gole, Robert Nelson, Andrew Davis, Greg Kroah-Hartman,
	David Gibson, Luca Ceresoli, Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

Hi Ajush,

On Mon, 24 Feb 2025 15:39:41 +0530
Ayush Singh <ayush@beagleboard.org> wrote:

> On 2/24/25 14:07, Geert Uytterhoeven wrote:
> 
> > Hi Ayush,
> >
> > On Sat, 22 Feb 2025 at 21:14, Ayush Singh <ayush@beagleboard.org> wrote:  
> >> # Challenges
> >>
> >> ## Security
> >>
> >> The concerns regarding security seemed to show up in the other
> >> proposals. There was a proposal to have a devicetree property to
> >> allow/deny the application of overlays in some nodes, with default being
> >> deny. Was it insufficient?  
> > This is the most important issue: using DT overlays, you can change
> > about anything.  There is no protection yet to limit this to e.g. the
> > expansion connectors on your board.
> > This is what the various WIP "connector" abstractions are trying
> > to solve.  
> 
> Thanks for clarifying. However, as I mentioned above, there are usecases 
> for dynamic overlays outside of connectors. Specifically, for the 
> usecase of connecting random sensors to board pins. I do agree that any 
> fairly well specified connector should probably have it's own drivers 
> rather than using a generic userspace API.
> 
> Would it be enough to use `aliases` to specify the nodes where an 
> overlay can be applied as I have described here [0]. To further lock 
> down things, it might be possible to allow introducing indirection or 
> scoping nodes of sort. Here is an example:
> 
> \ {
> 
> aliases {
> 
> export_node0 = &spi0_scoped;
> 
> };
> 
> };
> 
> &spi0 {
> 
> spi0_scoped {};
> 
> // Stuff already connected internally
> 
> };
> 
> Any children of `spi0_scoped` should be treated as devices directly 
> connected to `spi0` but should provided isolation for any changes that 
> userspace overlays can make.

For i2c bus, I introduced i2c bus extension feature.
  https://lore.kernel.org/all/20250205173918.600037-1-herve.codina@bootlin.com/

The series didn't received any feedback from i2c maintainer yet but I hope
to have soon.

The approach proposed the series could be adapted on spi busses too.

Do you think the bus extension approach could be used in your use cases ?

Best regards,
Hervé

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-24  8:37 ` Geert Uytterhoeven
  2025-02-24 10:09   ` Ayush Singh
@ 2025-03-05  4:27   ` David Gibson
  1 sibling, 0 replies; 12+ messages in thread
From: David Gibson @ 2025-03-05  4:27 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: Ayush Singh, xypron.glpk, Jason Kridner, Deepak Khatri, d-gole,
	Robert Nelson, Andrew Davis, Greg Kroah-Hartman, Luca Ceresoli,
	Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

[-- Attachment #1: Type: text/plain, Size: 1635 bytes --]

On Mon, Feb 24, 2025 at 09:37:56AM +0100, Geert Uytterhoeven wrote:
> Hi Ayush,
> 
> On Sat, 22 Feb 2025 at 21:14, Ayush Singh <ayush@beagleboard.org> wrote:
> > # Challenges
> >
> > ## Security
> >
> > The concerns regarding security seemed to show up in the other
> > proposals. There was a proposal to have a devicetree property to
> > allow/deny the application of overlays in some nodes, with default being
> > deny. Was it insufficient?
> 
> This is the most important issue: using DT overlays, you can change
> about anything.  There is no protection yet to limit this to e.g. the
> expansion connectors on your board.

Right.

> This is what the various WIP "connector" abstractions are trying
> to solve.

Exactly.

> > ## Memory Leaks
> >
> > Currently, updating/removing properties leaks memory. Was it one of the
> > reasons for the rejection of previous proposals?
> 
> IMO this is a minor issue. I am sure this can be improved upon.  We just
> need some way to keep track of which properties are part of the initial
> FDT (and thus can't be freed), and which were allocated dynamically.

Somewhat related to this, "unapplying" a DT overlay is not a natural
operation, but we need it for a sane update interface.  The normal way
of applying overlays is a lossy process.  Obviously, it's possible to
make it reversible by keeping around enough undo information, but it's
kind of a hack.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-24 10:09   ` Ayush Singh
  2025-02-24 13:29     ` Herve Codina
@ 2025-03-05  4:31     ` David Gibson
  2025-03-10  5:22       ` Ayush Singh
  1 sibling, 1 reply; 12+ messages in thread
From: David Gibson @ 2025-03-05  4:31 UTC (permalink / raw)
  To: Ayush Singh
  Cc: Geert Uytterhoeven, xypron.glpk, Jason Kridner, Deepak Khatri,
	d-gole, Robert Nelson, Andrew Davis, Greg Kroah-Hartman,
	Luca Ceresoli, Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

[-- Attachment #1: Type: text/plain, Size: 2111 bytes --]

On Mon, Feb 24, 2025 at 03:39:41PM +0530, Ayush Singh wrote:
> On 2/24/25 14:07, Geert Uytterhoeven wrote:
> 
> > Hi Ayush,
> > 
> > On Sat, 22 Feb 2025 at 21:14, Ayush Singh <ayush@beagleboard.org> wrote:
> > > # Challenges
> > > 
> > > ## Security
> > > 
> > > The concerns regarding security seemed to show up in the other
> > > proposals. There was a proposal to have a devicetree property to
> > > allow/deny the application of overlays in some nodes, with default being
> > > deny. Was it insufficient?
> > This is the most important issue: using DT overlays, you can change
> > about anything.  There is no protection yet to limit this to e.g. the
> > expansion connectors on your board.
> > This is what the various WIP "connector" abstractions are trying
> > to solve.
> 
> Thanks for clarifying. However, as I mentioned above, there are usecases for
> dynamic overlays outside of connectors. Specifically, for the usecase of
> connecting random sensors to board pins. I do agree that any fairly well
> specified connector should probably have it's own drivers rather than using
> a generic userspace API.

I'm not sure that's just due to an insuffuciently broad conception of
what a "connector" might be.  Note that to justify a dynamic overlay
interface specifically you need to have *both*
  1) a need to update *anywhere* in the device tree and
  2) to do so at runtime, under userspace control

It's kind of hard to see why you'd need (2) in cases that don't at
some physical level involve a "connector".. in which case (1) is hard
to justify.

How are these sensors being connected to random board pins?  If it's
because those pins are exposed on some header, then it seems like it
ought to fall within the definition of a connector.  If someone is
just soldering onto them, it seems like an semi-permanent change that
would be better handled at boot time.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-03-05  4:31     ` David Gibson
@ 2025-03-10  5:22       ` Ayush Singh
  0 siblings, 0 replies; 12+ messages in thread
From: Ayush Singh @ 2025-03-10  5:22 UTC (permalink / raw)
  To: David Gibson
  Cc: Geert Uytterhoeven, xypron.glpk, Jason Kridner, Deepak Khatri,
	d-gole, Robert Nelson, Andrew Davis, Greg Kroah-Hartman,
	Luca Ceresoli, Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

On 3/5/25 10:01, David Gibson wrote:

> On Mon, Feb 24, 2025 at 03:39:41PM +0530, Ayush Singh wrote:
>> On 2/24/25 14:07, Geert Uytterhoeven wrote:
>>
>>> Hi Ayush,
>>>
>>> On Sat, 22 Feb 2025 at 21:14, Ayush Singh <ayush@beagleboard.org> wrote:
>>>> # Challenges
>>>>
>>>> ## Security
>>>>
>>>> The concerns regarding security seemed to show up in the other
>>>> proposals. There was a proposal to have a devicetree property to
>>>> allow/deny the application of overlays in some nodes, with default being
>>>> deny. Was it insufficient?
>>> This is the most important issue: using DT overlays, you can change
>>> about anything.  There is no protection yet to limit this to e.g. the
>>> expansion connectors on your board.
>>> This is what the various WIP "connector" abstractions are trying
>>> to solve.
>> Thanks for clarifying. However, as I mentioned above, there are usecases for
>> dynamic overlays outside of connectors. Specifically, for the usecase of
>> connecting random sensors to board pins. I do agree that any fairly well
>> specified connector should probably have it's own drivers rather than using
>> a generic userspace API.
> I'm not sure that's just due to an insuffuciently broad conception of
> what a "connector" might be.  Note that to justify a dynamic overlay
> interface specifically you need to have *both*
>    1) a need to update *anywhere* in the device tree and
>    2) to do so at runtime, under userspace control
>
> It's kind of hard to see why you'd need (2) in cases that don't at
> some physical level involve a "connector".. in which case (1) is hard
> to justify.
>
> How are these sensors being connected to random board pins?  If it's
> because those pins are exposed on some header, then it seems like it
> ought to fall within the definition of a connector.  If someone is
> just soldering onto them, it seems like an semi-permanent change that
> would be better handled at boot time.


I see. It seems my perception of connector was a bit too narrow. 
Certainly, treating the whole board header as a connector would 
certainly be a better solution, since it will also allow great control 
using a dedicated driver. Thanks for the insight.


Ayush Singh


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-02-22 20:13 [Question] Status of user-space dynamic overlays API Ayush Singh
                   ` (2 preceding siblings ...)
  2025-02-24 11:24 ` Luca Ceresoli
@ 2025-04-30 10:18 ` Ayush Singh
  2025-04-30 13:07   ` Luca Ceresoli
  3 siblings, 1 reply; 12+ messages in thread
From: Ayush Singh @ 2025-04-30 10:18 UTC (permalink / raw)
  To: xypron.glpk, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Geert Uytterhoeven, Greg Kroah-Hartman,
	David Gibson, Luca Ceresoli, Pantelis Antoniou
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

On 2/23/25 01:43, Ayush Singh wrote:

> Hello everyone.
>
> I have been looking at ways to do runtime devicetree overlay 
> application, and was just wondering what the current status of the 
> different proposals [0], [1] were. They seem to be quite old and I 
> think they were already rejected, but due to all the broken links, I 
> am not really sure about the exact reasons. Also, maybe we now have 
> the solutions to some the blockers at the time.
>
>
> Let me fist go over some of the use cases where I think dynamic 
> devicetree overlays can be useful. I am mostly interested in their use 
> in single board computers like PocketBeagle 2 [2], Raspberry Pi [3], etc.
>
>
> # Uses
>
> ## Dynamic Pin muxing
>
> A lot of SBC's aimed for creating hardware projects expose headers, 
> where each pin can be used for multiple things like GPIO, I2C, PWM, 
> etc, depending on the pinmux. I think Raspberry Pi has it's own 
> solution to do userspace pinmux, but if userspace devicetree 
> application was a thing, it could probably be used for this. 
> Additionally, being able to use dynamic devicetree overlays for pin 
> muxing would allow much easier transition to use proper device trees 
> during production.
>
>
> ## Dynamic Sensors/Devices
>
> Using devices such as sensors, external ADCs, EEPROMs, etc are also a 
> common usecase in SBC's. A lot of current solutions seem to be 
> designed around using user-space drivers in such cases. This is a bit 
> of a shame since Linux kernel already has drivers for a lot of these 
> drivers, and they are probably going to be of higher quality than most 
> user space drivers.
>
>
> # Challenges
>
> ## Security
>
> The concerns regarding security seemed to show up in the other 
> proposals. There was a proposal to have a devicetree property to 
> allow/deny the application of overlays in some nodes, with default 
> being deny. Was it insufficient?
>
>
> ## Memory Leaks
>
> Currently, updating/removing properties leaks memory. Was it one of 
> the reasons for the rejection of previous proposals?
>
>
> Maybe kernel already has some solutions more suited to my usecase that 
> I am unware of?
>
>
> [0]: 
> https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t
>
> [1]: 
> https://lore.kernel.org/all/20161220190455.25115-1-xypron.glpk@gmx.de/
>
> [2]: https://www.beagleboard.org/boards/pocketbeagle-2
>
> [3]: https://www.raspberrypi.com/
>
>
> Best Regards,
>
> Ayush Singh
>

Just trying to consolidate the discussion. Feel free to correct anything 
wrong.


- Rather a generic global overlay solution, a driver per connector 
should be used.

- The board headers (e.g. PocketBeagle 2) should be treated as a single 
connector and any peripherals on it can be treated as an addon-board.


Best Regards,

Ayush Singh


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Question] Status of user-space dynamic overlays API
  2025-04-30 10:18 ` Ayush Singh
@ 2025-04-30 13:07   ` Luca Ceresoli
  0 siblings, 0 replies; 12+ messages in thread
From: Luca Ceresoli @ 2025-04-30 13:07 UTC (permalink / raw)
  To: Ayush Singh
  Cc: xypron.glpk, Jason Kridner, Deepak Khatri, d-gole, Robert Nelson,
	Andrew Davis, Geert Uytterhoeven, Greg Kroah-Hartman,
	David Gibson, Pantelis Antoniou,
	open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS

Hello Ayush,

On Wed, 30 Apr 2025 15:48:27 +0530
Ayush Singh <ayush@beagleboard.org> wrote:

> On 2/23/25 01:43, Ayush Singh wrote:
> 
> > Hello everyone.
> >
> > I have been looking at ways to do runtime devicetree overlay 
> > application, and was just wondering what the current status of the 
> > different proposals [0], [1] were. They seem to be quite old and I 
> > think they were already rejected, but due to all the broken links, I 
> > am not really sure about the exact reasons. Also, maybe we now have 
> > the solutions to some the blockers at the time.
> >
> >
> > Let me fist go over some of the use cases where I think dynamic 
> > devicetree overlays can be useful. I am mostly interested in their use 
> > in single board computers like PocketBeagle 2 [2], Raspberry Pi [3], etc.
> >
> >
> > # Uses
> >
> > ## Dynamic Pin muxing
> >
> > A lot of SBC's aimed for creating hardware projects expose headers, 
> > where each pin can be used for multiple things like GPIO, I2C, PWM, 
> > etc, depending on the pinmux. I think Raspberry Pi has it's own 
> > solution to do userspace pinmux, but if userspace devicetree 
> > application was a thing, it could probably be used for this. 
> > Additionally, being able to use dynamic devicetree overlays for pin 
> > muxing would allow much easier transition to use proper device trees 
> > during production.
> >
> >
> > ## Dynamic Sensors/Devices
> >
> > Using devices such as sensors, external ADCs, EEPROMs, etc are also a 
> > common usecase in SBC's. A lot of current solutions seem to be 
> > designed around using user-space drivers in such cases. This is a bit 
> > of a shame since Linux kernel already has drivers for a lot of these 
> > drivers, and they are probably going to be of higher quality than most 
> > user space drivers.
> >
> >
> > # Challenges
> >
> > ## Security
> >
> > The concerns regarding security seemed to show up in the other 
> > proposals. There was a proposal to have a devicetree property to 
> > allow/deny the application of overlays in some nodes, with default 
> > being deny. Was it insufficient?
> >
> >
> > ## Memory Leaks
> >
> > Currently, updating/removing properties leaks memory. Was it one of 
> > the reasons for the rejection of previous proposals?
> >
> >
> > Maybe kernel already has some solutions more suited to my usecase that 
> > I am unware of?
> >
> >
> > [0]: 
> > https://lore.kernel.org/all/1417605808-23327-1-git-send-email-pantelis.antoniou@konsulko.com/#t
> >
> > [1]: 
> > https://lore.kernel.org/all/20161220190455.25115-1-xypron.glpk@gmx.de/
> >
> > [2]: https://www.beagleboard.org/boards/pocketbeagle-2
> >
> > [3]: https://www.raspberrypi.com/
> >
> >
> > Best Regards,
> >
> > Ayush Singh
> >  
> 
> Just trying to consolidate the discussion. Feel free to correct anything 
> wrong.
> 
> 
> - Rather a generic global overlay solution, a driver per connector 
> should be used.
> 
> - The board headers (e.g. PocketBeagle 2) should be treated as a single 
> connector and any peripherals on it can be treated as an addon-board.

I agree with both points.

To the best of my knowledge this is also the position that everybody
agrees on, except Andrew Davis.

That said, I think this topic desperately needs some official feedback
from device tree and dt-schema maintainers: about export-symbols in the
very first place, and about i2c-bus-extension immediately after.
Otherwise we're keeping on circling around the same ideas after having
discussed and refined them so many times.

Luca

-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2025-04-30 13:07 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-22 20:13 [Question] Status of user-space dynamic overlays API Ayush Singh
2025-02-22 20:31 ` Heinrich Schuchardt
2025-02-24  5:58   ` Ayush Singh
2025-02-24  8:37 ` Geert Uytterhoeven
2025-02-24 10:09   ` Ayush Singh
2025-02-24 13:29     ` Herve Codina
2025-03-05  4:31     ` David Gibson
2025-03-10  5:22       ` Ayush Singh
2025-03-05  4:27   ` David Gibson
2025-02-24 11:24 ` Luca Ceresoli
2025-04-30 10:18 ` Ayush Singh
2025-04-30 13:07   ` Luca Ceresoli

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).