Node name & property name collusion - json/yaml implications

Node name & property name collusion - json/yaml implications

[Kumar Gala]

There’s an old thread about this from 2016:

https://www.spinics.net/lists/devicetree-spec/msg00296.html

In which its clearly stated that node names and property names technically can be the same.  However, as we’ve starting utilizing tooling like JSON for validation, does it make sense to maintain this and should we update the specification to require that a node name and property name at the same hierarchy in the tree is not allowed?

Otherwise we get into fun situations like, being valid DTS, but invalid YAML, and various tools not working correct as the YAML loaders have to pick one or there the other version of ‘foo’:

[Example is from Marti]

/dts-v1/;
/ {
	foo;
	foo {
		bar = <0>;
	};
};

(Using pyYAML)

$ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, yaml; print(yaml.safe_load(sys.stdin.read()))’
[{'foo': {'bar': [[0]]}}]

(Using ruamel, w/allow_duplicate_keys as dt-schema does)

$ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, ruamel.yaml; yaml = ruamel.yaml.YAML(typ="safe"); yaml.allow_duplicate_keys = True; print(yaml.load(sys.stdin.read()))’
[{'foo': True}]

- k

Re: Node name & property name collusion - json/yaml implications

[Rob Herring]

On 2/5/21 6:57 AM, Kumar Gala wrote:
> There’s an old thread about this from 2016:
>
> https://www.spinics.net/lists/devicetree-spec/msg00296.html
>
> In which its clearly stated that node names and property names technically can be the same.  However, as we’ve starting utilizing tooling like JSON for validation, does it make sense to maintain this and should we update the specification to require that a node name and property name at the same hierarchy in the tree is not allowed?
>

IMO, yes. We are effectively doing that already given some of the
tooling doesn't really support it.

We've removed the few, rare cases in the kernel tree dts files.

> Otherwise we get into fun situations like, being valid DTS, but invalid YAML, and various tools not working correct as the YAML loaders have to pick one or there the other version of ‘foo’:
>
> [Example is from Marti]
>
> /dts-v1/;
> / {
>       foo;
>       foo {
>               bar = <0>;
>       };
> };
>
> (Using pyYAML)
>
> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, yaml; print(yaml.safe_load(sys.stdin.read()))’
> [{'foo': {'bar': [[0]]}}]
>
> (Using ruamel, w/allow_duplicate_keys as dt-schema does)

That reminds me, I need to go turn that off.

>
> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, ruamel.yaml; yaml = ruamel.yaml.YAML(typ="safe"); yaml.allow_duplicate_keys = True; print(yaml.load(sys.stdin.read()))’
> [{'foo': True}>
> - k
>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: Node name & property name collusion - json/yaml implications

[Kumar Gala]

> On Feb 5, 2021, at 11:25 AM, Rob Herring <rob.herring-5wv7dgnIgG8@public.gmane.org> wrote:
> 
> On 2/5/21 6:57 AM, Kumar Gala wrote:
>> There’s an old thread about this from 2016:
>> 
>> https://www.spinics.net/lists/devicetree-spec/msg00296.html
>> 
>> In which its clearly stated that node names and property names technically can be the same.  However, as we’ve starting utilizing tooling like JSON for validation, does it make sense to maintain this and should we update the specification to require that a node name and property name at the same hierarchy in the tree is not allowed?
>> 
> 
> IMO, yes. We are effectively doing that already given some of the
> tooling doesn't really support it.
> 
> We've removed the few, rare cases in the kernel tree dts files.
> 
>> Otherwise we get into fun situations like, being valid DTS, but invalid YAML, and various tools not working correct as the YAML loaders have to pick one or there the other version of ‘foo’:
>> 
>> [Example is from Marti]
>> 
>> /dts-v1/;
>> / {
>>      foo;
>>      foo {
>>              bar = <0>;
>>      };
>> };
>> 
>> (Using pyYAML)
>> 
>> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, yaml; print(yaml.safe_load(sys.stdin.read()))’
>> [{'foo': {'bar': [[0]]}}]
>> 
>> (Using ruamel, w/allow_duplicate_keys as dt-schema does)
> 
> That reminds me, I need to go turn that off.

Was going to send you a patch to that, but sounds like you might get there first :)

>> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, ruamel.yaml; yaml = ruamel.yaml.YAML(typ="safe"); yaml.allow_duplicate_keys = True; print(yaml.load(sys.stdin.read()))’
>> [{'foo': True}>
>> - k

- k

Re: Node name & property name collusion - json/yaml implications

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
> There’s an old thread about this from 2016:
> 
> https://www.spinics.net/lists/devicetree-spec/msg00296.html

That is not the only case of this - IIRC old Apple machine device
trees had both an 'l2-cache' property and an 'l2-cache' node under
each CPU node.

> In which its clearly stated that node names and property names technically can be the same.  However, as we’ve starting utilizing tooling like JSON for validation, does it make sense to maintain this and should we update the specification to require that a node name and property name at the same hierarchy in the tree is not allowed?
> 
> Otherwise we get into fun situations like, being valid DTS, but invalid YAML, and various tools not working correct as the YAML loaders have to pick one or there the other version of ‘foo’:
> 
> [Example is from Marti]
> 
> /dts-v1/;
> / {
> 	foo;
> 	foo {
> 		bar = <0>;
> 	};
> };
> 
> (Using pyYAML)
> 
> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, yaml; print(yaml.safe_load(sys.stdin.read()))’
> [{'foo': {'bar': [[0]]}}]
> 
> (Using ruamel, w/allow_duplicate_keys as dt-schema does)
> 
> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, ruamel.yaml; yaml = ruamel.yaml.YAML(typ="safe"); yaml.allow_duplicate_keys = True; print(yaml.load(sys.stdin.read()))’
> [{'foo': True}]
> 
> - k

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Node name & property name collusion - json/yaml implications

[Kumar Gala]

> On Feb 6, 2021, at 1:11 AM, David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> 
> On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
>> There’s an old thread about this from 2016:
>> 
>> https://www.spinics.net/lists/devicetree-spec/msg00296.html
> 
> That is not the only case of this - IIRC old Apple machine device
> trees had both an 'l2-cache' property and an 'l2-cache' node under
> each CPU node.

Sure, but I assume this is an OpenFirmware machine, so does dts need to maintain support for it?

> 
>> In which its clearly stated that node names and property names technically can be the same.  However, as we’ve starting utilizing tooling like JSON for validation, does it make sense to maintain this and should we update the specification to require that a node name and property name at the same hierarchy in the tree is not allowed?
>> 
>> Otherwise we get into fun situations like, being valid DTS, but invalid YAML, and various tools not working correct as the YAML loaders have to pick one or there the other version of ‘foo’:
>> 
>> [Example is from Marti]
>> 
>> /dts-v1/;
>> / {
>> 	foo;
>> 	foo {
>> 		bar = <0>;
>> 	};
>> };
>> 
>> (Using pyYAML)
>> 
>> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, yaml; print(yaml.safe_load(sys.stdin.read()))’
>> [{'foo': {'bar': [[0]]}}]
>> 
>> (Using ruamel, w/allow_duplicate_keys as dt-schema does)
>> 
>> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, ruamel.yaml; yaml = ruamel.yaml.YAML(typ="safe"); yaml.allow_duplicate_keys = True; print(yaml.load(sys.stdin.read()))’
>> [{'foo': True}]
>> 
>> - k
> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson

Re: Node name & property name collusion - json/yaml implications

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]

On Mon, Feb 08, 2021 at 08:39:55AM -0600, Kumar Gala wrote:
> 
> 
> > On Feb 6, 2021, at 1:11 AM, David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > 
> > On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
> >> There’s an old thread about this from 2016:
> >> 
> >> https://www.spinics.net/lists/devicetree-spec/msg00296.html
> > 
> > That is not the only case of this - IIRC old Apple machine device
> > trees had both an 'l2-cache' property and an 'l2-cache' node under
> > each CPU node.
> 
> Sure, but I assume this is an OpenFirmware machine, so does dts need
> to maintain support for it?

dtc needs to maintain support for it, yes, because dtc is still useful
for investigating device tree related things on OF machines, not just
for building flattened trees from scratch.

> 
> > 
> >> In which its clearly stated that node names and property names technically can be the same.  However, as we’ve starting utilizing tooling like JSON for validation, does it make sense to maintain this and should we update the specification to require that a node name and property name at the same hierarchy in the tree is not allowed?
> >> 
> >> Otherwise we get into fun situations like, being valid DTS, but invalid YAML, and various tools not working correct as the YAML loaders have to pick one or there the other version of ‘foo’:
> >> 
> >> [Example is from Marti]
> >> 
> >> /dts-v1/;
> >> / {
> >> 	foo;
> >> 	foo {
> >> 		bar = <0>;
> >> 	};
> >> };
> >> 
> >> (Using pyYAML)
> >> 
> >> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, yaml; print(yaml.safe_load(sys.stdin.read()))’
> >> [{'foo': {'bar': [[0]]}}]
> >> 
> >> (Using ruamel, w/allow_duplicate_keys as dt-schema does)
> >> 
> >> $ dtc -I dts -O yaml ./foo.dts | python -c 'import sys, ruamel.yaml; yaml = ruamel.yaml.YAML(typ="safe"); yaml.allow_duplicate_keys = True; print(yaml.load(sys.stdin.read()))’
> >> [{'foo': True}]
> >> 
> >> - k
> > 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Node name & property name collusion - json/yaml implications

[Rob Herring]

On Sat, Feb 6, 2021 at 1:26 AM David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
>
> On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
> > There’s an old thread about this from 2016:
> >
> > https://www.spinics.net/lists/devicetree-spec/msg00296.html
>
> That is not the only case of this - IIRC old Apple machine device
> trees had both an 'l2-cache' property and an 'l2-cache' node under
> each CPU node.

According to a dump I have, '/cpus/PowerPC,G4' has 'l2-cache' property
and '/cpus/PowerPC,G4/PowerPC,G4' node has a node name of 'l2-cache'.
So this would actually work with yaml if we brought back explicit
'name' properties. But this highlights we've already diverged from
OpenFirmware.

Specs evolve all the time deprecating old behavior. I don't see this
as any different. We can debate deprecating vs. outright changing the
spec. It's so rare, do we need to even bother deprecating it?

Rob

Re: Node name & property name collusion - json/yaml implications

[Arnd Bergmann]

On Mon, Feb 8, 2021 at 4:01 PM Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>
> On Sat, Feb 6, 2021 at 1:26 AM David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> >
> > On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
> > > There’s an old thread about this from 2016:
> > >
> > > https://www.spinics.net/lists/devicetree-spec/msg00296.html
> >
> > That is not the only case of this - IIRC old Apple machine device
> > trees had both an 'l2-cache' property and an 'l2-cache' node under
> > each CPU node.
>
> According to a dump I have, '/cpus/PowerPC,G4' has 'l2-cache' property
> and '/cpus/PowerPC,G4/PowerPC,G4' node has a node name of 'l2-cache'.
> So this would actually work with yaml if we brought back explicit
> 'name' properties. But this highlights we've already diverged from
> OpenFirmware.

For reference, the new Mac Mini has a "pmgr" node that has both
a property clpc=<0x3> and a child node named 'clpc'.

         Arnd

Re: Node name & property name collusion - json/yaml implications

[Rob Herring]

On Mon, Feb 8, 2021 at 9:47 AM Arnd Bergmann <arnd-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>
> On Mon, Feb 8, 2021 at 4:01 PM Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> >
> > On Sat, Feb 6, 2021 at 1:26 AM David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6iEOx6SR5NKn@public.gmane.orgu> wrote:
> > >
> > > On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
> > > > There’s an old thread about this from 2016:
> > > >
> > > > https://www.spinics.net/lists/devicetree-spec/msg00296.html
> > >
> > > That is not the only case of this - IIRC old Apple machine device
> > > trees had both an 'l2-cache' property and an 'l2-cache' node under
> > > each CPU node.
> >
> > According to a dump I have, '/cpus/PowerPC,G4' has 'l2-cache' property
> > and '/cpus/PowerPC,G4/PowerPC,G4' node has a node name of 'l2-cache'.
> > So this would actually work with yaml if we brought back explicit
> > 'name' properties. But this highlights we've already diverged from
> > OpenFirmware.
>
> For reference, the new Mac Mini has a "pmgr" node that has both
> a property clpc=<0x3> and a child node named 'clpc'.

That's the tip of the iceberg for the new Apple stuff. I have no
interest in supporting a 3rd firmware interface as that is what it is.
It may share some heritage with DT, but it's 15 years of Apple
evolving their own thing in a vacuum.

Rob

Re: Node name & property name collusion - json/yaml implications

[Kumar Gala]

> On Feb 8, 2021, at 10:00 AM, Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> 
> On Mon, Feb 8, 2021 at 9:47 AM Arnd Bergmann <arnd-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>> 
>> On Mon, Feb 8, 2021 at 4:01 PM Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>>> 
>>> On Sat, Feb 6, 2021 at 1:26 AM David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
>>>> 
>>>> On Fri, Feb 05, 2021 at 06:57:54AM -0600, Kumar Gala wrote:
>>>>> There’s an old thread about this from 2016:
>>>>> 
>>>>> https://www.spinics.net/lists/devicetree-spec/msg00296.html
>>>> 
>>>> That is not the only case of this - IIRC old Apple machine device
>>>> trees had both an 'l2-cache' property and an 'l2-cache' node under
>>>> each CPU node.
>>> 
>>> According to a dump I have, '/cpus/PowerPC,G4' has 'l2-cache' property
>>> and '/cpus/PowerPC,G4/PowerPC,G4' node has a node name of 'l2-cache'.
>>> So this would actually work with yaml if we brought back explicit
>>> 'name' properties. But this highlights we've already diverged from
>>> OpenFirmware.
>> 
>> For reference, the new Mac Mini has a "pmgr" node that has both
>> a property clpc=<0x3> and a child node named 'clpc'.
> 
> That's the tip of the iceberg for the new Apple stuff. I have no
> interest in supporting a 3rd firmware interface as that is what it is.
> It may share some heritage with DT, but it's 15 years of Apple
> evolving their own thing in a vacuum.
> 
> Rob

I agree with Rob here.  I don’t think we should contort the directions we want to take devicetree spec to an entity that is not engaged with the DT community.

- k