Overlays and boolean properties

Overlays and boolean properties

[Phil Elwell]

Boolean properties are defined as being properties with no content, that
are true if present and false if absent. They pose a problem for DT
overlays, since the proposed (and widely used) overlay mechanism does
not allow for properties (or nodes) to be deleted; overlays can only
make a false property true, so boolean properties are effectively
monostable - once true they become immutable.

The standard DT syntax includes /delete-property/ and /delete-node/
directives that do what you would expect from their names, but that
facility is not available to overlays. There is no FDT node that
represents the deletion - the directives are acted on immediately - so
we would need some extra markup - say __delete_property__ and
__delete_node__ - to hold the names of items to be deleted.

Before I take this further, does anybody have any thoughts on the idea?

Phil

Re: Overlays and boolean properties

[Pantelis Antoniou]

Hi Phil,

> On Nov 29, 2016, at 15:06 , Phil Elwell <phil-FnsA7b+Nu9XbIbC87yuRow@public.gmane.org> wrote:
> 
> Boolean properties are defined as being properties with no content, that
> are true if present and false if absent. They pose a problem for DT
> overlays, since the proposed (and widely used) overlay mechanism does
> not allow for properties (or nodes) to be deleted; overlays can only
> make a false property true, so boolean properties are effectively
> monostable - once true they become immutable.
> 
> The standard DT syntax includes /delete-property/ and /delete-node/
> directives that do what you would expect from their names, but that
> facility is not available to overlays. There is no FDT node that
> represents the deletion - the directives are acted on immediately - so
> we would need some extra markup - say __delete_property__ and
> __delete_node__ - to hold the names of items to be deleted.
> 
> Before I take this further, does anybody have any thoughts on the idea?
> 

The original patchset did support removing properties (by prefixing them with -).

I can revive that if we have consensus about the format/method.

> Phil
> 

Regards

— Pantelis

Re: Overlays and boolean properties

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2345 bytes --]

On Tue, Nov 29, 2016 at 03:10:40PM +0200, Pantelis Antoniou wrote:
> Hi Phil,
> 
> > On Nov 29, 2016, at 15:06 , Phil Elwell <phil-FnsA7b+Nu9XbIbC87yuRow@public.gmane.org> wrote:
> > 
> > Boolean properties are defined as being properties with no content, that
> > are true if present and false if absent. They pose a problem for DT
> > overlays, since the proposed (and widely used) overlay mechanism does
> > not allow for properties (or nodes) to be deleted; overlays can only
> > make a false property true, so boolean properties are effectively
> > monostable - once true they become immutable.
> > 
> > The standard DT syntax includes /delete-property/ and /delete-node/
> > directives that do what you would expect from their names, but that
> > facility is not available to overlays. There is no FDT node that
> > represents the deletion - the directives are acted on immediately

Uh.. only partially true.  They're acted on during the compile run,
but not during the parse.  dtc does have an internal representation of
node or property deletions that gets resolved when we combine the
fragments in the source file.
> > - so
> > we would need some extra markup - say __delete_property__ and
> > __delete_node__ - to hold the names of items to be deleted.

So, in principle this wouldn't be that hard - we'd just need to
translate dtc's internal representation into a representation in the
dtb.  That could be done with special properties, or with new opcodes
at the dtb encoding level.

> > Before I take this further, does anybody have any thoughts on the idea?

So.. the first question, is do we have a pressing use case for this?
dtbos can (apart from this) alter anything in a base tree, but doing
so isn't often a good idea.  Usually they'll just add new nodes and
properties.

> The original patchset did support removing properties (by prefixing them with -).
> 
> I can revive that if we have consensus about the format/method.

On the whole, I'd prefer not to see extensions of the existing overlay
format - instead I'd like to see focus on a new and better thought out
connector format.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Overlays and boolean properties

[Phil Elwell]

Hi David,

On 30/11/2016 03:47, David Gibson wrote:
> On Tue, Nov 29, 2016 at 03:10:40PM +0200, Pantelis Antoniou wrote:
>> Hi Phil,
>>
>>> On Nov 29, 2016, at 15:06 , Phil Elwell <phil-FnsA7b+Nu9XbIbC87yuRow@public.gmane.org> wrote:
>>>
>>> Boolean properties are defined as being properties with no content, that
>>> are true if present and false if absent. They pose a problem for DT
>>> overlays, since the proposed (and widely used) overlay mechanism does
>>> not allow for properties (or nodes) to be deleted; overlays can only
>>> make a false property true, so boolean properties are effectively
>>> monostable - once true they become immutable.
>>>
>>> The standard DT syntax includes /delete-property/ and /delete-node/
>>> directives that do what you would expect from their names, but that
>>> facility is not available to overlays. There is no FDT node that
>>> represents the deletion - the directives are acted on immediately
> Uh.. only partially true.  They're acted on during the compile run,
> but not during the parse.  dtc does have an internal representation of
> node or property deletions that gets resolved when we combine the
> fragments in the source file.
>>> - so
>>> we would need some extra markup - say __delete_property__ and
>>> __delete_node__ - to hold the names of items to be deleted.
> So, in principle this wouldn't be that hard - we'd just need to
> translate dtc's internal representation into a representation in the
> dtb.  That could be done with special properties, or with new opcodes
> at the dtb encoding level.
>
>>> Before I take this further, does anybody have any thoughts on the idea?
> So.. the first question, is do we have a pressing use case for this?
> dtbos can (apart from this) alter anything in a base tree, but doing
> so isn't often a good idea.  Usually they'll just add new nodes and
> properties.

This is more of a real world example than a pressing use case, since the
number of people affected is small and there is workaround, albeit an
ugly one.

The Raspberry Pi SoCs have two SD interfaces - one SDIO-capable and one
not. On the Pi3B the SDIO-capable interface is used to drive the WiFi
interface. This WiFi interface doesn't like being continuously prodded
to see if it is still there, so the base DTB sets the "non-removable"
boolean property. Not all Pi3 users want WiFi, preferring instead to
wire up an MMC device or second SD slot via the expansion header. A DT
overlay allows them to modify the pin functions to achieve this, but the
overlay can't (easily) remove the aptly-named "non-removable". The ugly
workaround is to disable the original MMC node and create a clone with a
different name and without the unwanted property.

>> The original patchset did support removing properties (by prefixing them with -).
>>
>> I can revive that if we have consensus about the format/method.
> On the whole, I'd prefer not to see extensions of the existing overlay
> format - instead I'd like to see focus on a new and better thought out
> connector format.
Will this connector mechanism allow a populated DT to be modified, or
would you always have to start with an "empty" board and only add that
which you want? I like the flexibility to be able to perform arbitrary
DT modifications (except deletions) - it has been very useful for
testing and performing staggered rollouts of new functionality.

Phil

Re: Overlays and boolean properties

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 4428 bytes --]

On Wed, Nov 30, 2016 at 09:29:22AM +0000, Phil Elwell wrote:
> Hi David,
> 
> On 30/11/2016 03:47, David Gibson wrote:
> > On Tue, Nov 29, 2016 at 03:10:40PM +0200, Pantelis Antoniou wrote:
> >> Hi Phil,
> >>
> >>> On Nov 29, 2016, at 15:06 , Phil Elwell <phil-FnsA7b+Nu9XbIbC87yuRow@public.gmane.org> wrote:
> >>>
> >>> Boolean properties are defined as being properties with no content, that
> >>> are true if present and false if absent. They pose a problem for DT
> >>> overlays, since the proposed (and widely used) overlay mechanism does
> >>> not allow for properties (or nodes) to be deleted; overlays can only
> >>> make a false property true, so boolean properties are effectively
> >>> monostable - once true they become immutable.
> >>>
> >>> The standard DT syntax includes /delete-property/ and /delete-node/
> >>> directives that do what you would expect from their names, but that
> >>> facility is not available to overlays. There is no FDT node that
> >>> represents the deletion - the directives are acted on immediately
> > Uh.. only partially true.  They're acted on during the compile run,
> > but not during the parse.  dtc does have an internal representation of
> > node or property deletions that gets resolved when we combine the
> > fragments in the source file.
> >>> - so
> >>> we would need some extra markup - say __delete_property__ and
> >>> __delete_node__ - to hold the names of items to be deleted.
> > So, in principle this wouldn't be that hard - we'd just need to
> > translate dtc's internal representation into a representation in the
> > dtb.  That could be done with special properties, or with new opcodes
> > at the dtb encoding level.
> >
> >>> Before I take this further, does anybody have any thoughts on the idea?
> > So.. the first question, is do we have a pressing use case for this?
> > dtbos can (apart from this) alter anything in a base tree, but doing
> > so isn't often a good idea.  Usually they'll just add new nodes and
> > properties.
> 
> This is more of a real world example than a pressing use case, since the
> number of people affected is small and there is workaround, albeit an
> ugly one.
> 
> The Raspberry Pi SoCs have two SD interfaces - one SDIO-capable and one
> not. On the Pi3B the SDIO-capable interface is used to drive the WiFi
> interface. This WiFi interface doesn't like being continuously prodded
> to see if it is still there, so the base DTB sets the "non-removable"
> boolean property. Not all Pi3 users want WiFi, preferring instead to
> wire up an MMC device or second SD slot via the expansion header. A DT
> overlay allows them to modify the pin functions to achieve this, but the
> overlay can't (easily) remove the aptly-named "non-removable". The ugly
> workaround is to disable the original MMC node and create a clone with a
> different name and without the unwanted property.

Ah, I see.  Yes, I can certainly see the use 

> >> The original patchset did support removing properties (by prefixing them with -).
> >>
> >> I can revive that if we have consensus about the format/method.
> > On the whole, I'd prefer not to see extensions of the existing overlay
> > format - instead I'd like to see focus on a new and better thought out
> > connector format.
> Will this connector mechanism allow a populated DT to be modified, or
> would you always have to start with an "empty" board and only add that
> which you want? I like the flexibility to be able to perform arbitrary
> DT modifications (except deletions) - it has been very useful for
> testing and performing staggered rollouts of new functionality.

In general, no, or at least connectors will only allow pieces of the
DT to be changed that have been allowed to change by the base DT.

The thing is overlays have really been used for two different
purposes: 1) for "production" use where we assemble the DT from
different pieces relevant for this board / system, 2) for testing and
debug where we make arbitrary runtime changes for experimentation.

I'd envisage connectors taking over function (1), but retaining a
mechanism similar to current overlays available (if discouraged) for
usage (2).

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]