Re: [RFC] Introducing yamldt, a yaml to dtb compiler

[David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 4689 bytes --]

On Thu, Jul 27, 2017 at 07:49:11PM +0300, Pantelis Antoniou wrote:
> Hi all,
> 
> This is a project I've been working on lately and it's finally in a
> usuable form.
> 
> I'm introducing yamldt.
> 
> A YAML to DT blob generator/compiler, utilizing a YAML schema that is
> functionaly equivalent to DTS and supports all DTS features.
> 
> yamldl parses a device tree description (source) file in YAML format and
> outputs a (bit-exact if the -C option is used) device tree blob.
> 
> A DT aware YAML schema is a good fit as a DTS syntax alternative.
> 
> YAML is a human-readable data serialization language, and is expressive
> enough to cover all DTS source features.
> 
> Simple YAML files are just key value pairs that are very easy to parse,
> even without using a formal YAML parser. For instance YAML in restricted
> environments may simple be appending a few lines of text in a given YAML
> file.
> 
> The parsers of YAML are very mature, as it has been released in 2001. It
> is in wide-spread use and schema validation tools are available. YAML
> support is available for every major programming language.
> 
> Data in YAML can easily be converted to/form other format that a
> particular tool that we may use in the future understands.
> 
> More importantly YAML offers (an optional) type information for each
> data, which is IMHO crucial for thorough validation and checking against
> device tree bindings (when they will be converted to a machine readable
> format, preferably YAML).
> 
> For more take a look here.
> 
> https://github.com/pantoniou/yamldt
> 
> I am eagerly awaiting for your comments.

Ok, technical comments here only; I addressthe procedural questions
brought up in the thread elsewhere.

First, there's a lot to like about YAML - if it had been as well known
when I wrote dtc, maybe we'd already be using it.  It was also the
frontrunner for a schema language in the various inconclusive threads
there have been on the topic.  It's been a little while since I read
up on YAML, so I may have forgotten some things about it.

I do have some doubts about this approach.

(1)

dts has its semantic model built closely around what dtb can
represent.  YAML (and JSON) have a different semantic model - in many
ways a better one than dtb (and IEEE1275), but that's not really the
point.  I wonder if having a source language which suggests the
possibility of things that can't actually be done in dtb will be
confusing.  The most obvious example is that any explicit type tags
will be stripped, of course, but there are others: nested list
structure can't be preserved in dtb, nor even what basic scalars are
in a list.  i.e. dtb couldn't tell the difference between:
	foo: [0, "\0\0\0\0"];
and
	foo: ["\0\0\0\0", 0];
	
There's also the fact that using YAML implicitly puts nodes and
properties into the namespace, which isn't the case in the dtb model.
Obviously you can simply ban having a property and subnode with the
same name (since that's good practice anyway), but it could be an
issue for decompiling or manipulating existing trees. I know there
have been device trees in the wild which had a property and subnode
with the same name in the same place (some old PowerPC based
Macintoshes, I think).

(2)

In the other direction there are several features of the dts format
I don't think you'll get for free with YAML - and it's not clear how
you would represent them there.  Obviously you *can* represent them -
it's a key value tree, so it can represent anything; whether it's
natural and readable is a different question.

YAML might have an equivalent of /incbin/, I'm not sure.  I'm pretty
sure it doesn't have integer expression evaluation, which is quite
useful in dts when combined with includes.  Likewise, how would you
tell a YAML based compiler what size to use when encoding a list of
integers - the equivalent of dtc's /bits/ directive.

(3)

It's not clear to me that preserving type information helps all that
much with validation.  You still have to validate against something,
so you need a schema.  And if you have a schema, you can get type and
structure information from there which will let you interpret the
untyped dt information.  That has the additional advantage that you
can also validate dtbs, which is a nice debugging feature when working
with some dtb that you've got from firmware or somewhere without any
dts/yaml/whatever.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]