Schemas for device trees

Schemas for device trees

[jonsmirl-Re5JQEeQqe8AvxtiuMwx3w]

I'm putting together a device tree for a new platform and when looking
at the other trees everything is kind of chaotic in regards to the
various subsystems.

Is the existing DT include mechanism powerful enough to describe
generic fragments? And we're just missing the core fragments for
generic SPI, MMC, audio, etc.

In the XML world you can make schemas that describe what is legally
allowed in an XML document. Device trees don't seem to any any kind of
schema mechanism describing what is and isn't legal to put in a tree.

Instead of recreating the entire world of schemas, has anyone looked
at using XML schemas to define what is a valid device tree and then
write a tool to validate the existing device trees?

A neat feature of XML schemas is the ability to include fragment
schemas. For example you'd have a base schema and then depending on
the core processor bring in schema fragments for mmc, audio, spi, etc.
 I still can't figure out the right way to put audio into a device
tree.

Another example, the voltage-ranges attribute for a spi mmc slot.
voltage-ranges = <3200 3400>; I had to go look at the source code for
this attribute to determine what the legal ranges were. A validating
schema would contain the legal ranges.

Aren't the files in Documentation/devicetree/bindings really just
schemas written in English with no automated way of checking the DTS
files against them?

-- 
Jon Smirl
jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

Re: Schemas for device trees

[Grant Likely]

On Wed, 28 Mar 2012 10:55:15 -0400, "jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org" <jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> I'm putting together a device tree for a new platform and when looking
> at the other trees everything is kind of chaotic in regards to the
> various subsystems.
> 
> Is the existing DT include mechanism powerful enough to describe
> generic fragments? And we're just missing the core fragments for
> generic SPI, MMC, audio, etc.

Yes, pretty much.  There is work going on now to define common
bus-specific bindings that can be used by all relevant drivers.

SPI is there
MMC is currently being discussed
audio is in progress

> 
> In the XML world you can make schemas that describe what is legally
> allowed in an XML document. Device trees don't seem to any any kind of
> schema mechanism describing what is and isn't legal to put in a tree.
> 
> Instead of recreating the entire world of schemas, has anyone looked
> at using XML schemas to define what is a valid device tree and then
> write a tool to validate the existing device trees?

Not that I know of.  dtc does minimal static validation of the output,
but it is not remotely enough.

> A neat feature of XML schemas is the ability to include fragment
> schemas. For example you'd have a base schema and then depending on
> the core processor bring in schema fragments for mmc, audio, spi, etc.
>  I still can't figure out the right way to put audio into a device
> tree.
> 
> Another example, the voltage-ranges attribute for a spi mmc slot.
> voltage-ranges = <3200 3400>; I had to go look at the source code for
> this attribute to determine what the legal ranges were. A validating
> schema would contain the legal ranges.
> 
> Aren't the files in Documentation/devicetree/bindings really just
> schemas written in English with no automated way of checking the DTS
> files against them?

Yes, they really are.  I've been thinking for a long time that it
would be much better to have a binding markup that can be parsed both
into human readable documentation and validation of device tree data.

Can you put together an example of what it would look like using the
xml schema?

g.

Re: Schemas for device trees

[jonsmirl-Re5JQEeQqe8AvxtiuMwx3w]

Check out the XML schema stuff but another possibility is simply
writing our own schema validation tools based on the existing DT
syntax.  What device trees really need is a schema language, how we
achieve it doesn't really matter.  A custom Perl app might be better
in the Linux kernel world than verbose XML which is aimed web authors.

The device tree gods would get together and develop this schema. The
rest of us would have to author device trees that validate against
this schema. If we can't make our hardware fit we would need to ask
for changes to be made to the master schema. No more rolling your own
DT syntax, everything would have to validate in order to be accepted
into the kernel.  The schema is going to codify all of those English
language bindings files.

----------------------------------------------------------

There is quite a bit to the XML schema language. A decent tutorial
covering the highlights is here:
http://www.w3schools.com/schema/schema_intro.asp

You can do everything in text files, but there are also GUI tools for
developing the schemas like XSD in Eclipse. XML schemas have been
around for about ten years and their use is widespread on the Internet
so there are lots of tools for working on them.

Two pieces are needed.

1) A fairly simple tool that converts DT syntax into XML.  This is
just a straight forward mapping. I don't think there is anything in DT
syntax that can't be mapped straight into XML.

2) Schemas that are probably authored in a XML schema GUI of some kind
(like Eclipse XSD).  These can get big and complex but just use a GUI
tool to work on them. These schemas describe the possible valid device
trees. Schemas can include other schemas, which lets you collect the
various subsystems together.

Once you have these two pieces you run a validation program than
compares the XML version of the DT to the schema.  It will kick out
errors. You probably already have a validator installed, try
'xmllint'.

The reference schema needed for validation can be kept in a specially
formatted comment at the top of the DTS file or we can modify the
device tree compiler to explicitly allow it.
/* xsi:schemaLocation="http://www.w3schools.com note.xsd" */
The schemas are commonly kept on a web sever and are fetched as
needed. Doing it that way makes it easy to change them and keep
everyone up to date.

Piece of a simple schema:

<xs:element name="note">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="to" type="xs:string"/>
      <xs:element name="from" type="xs:string"/>
      <xs:element name="heading" type="xs:string"/>
      <xs:element name="body" type="xs:string"/>
    </xs:sequence>
  </xs:complexType>
</xs:element>

Describes data like this:

<note>
<to>Joe</to>
<from>Bill</from>
</note>

If you feed it this, you'd get a validation error.

<note>
<author>Joe</author>
<from>Bill</from>
</note>


-- 
Jon Smirl
jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

Re: Schemas for device trees

[jonsmirl-Re5JQEeQqe8AvxtiuMwx3w]

i2c provides an example of where a schema would be useful.
>From the binding text files, I removed the common items

fsl-i2c
	i2c@1740 {
		clock-frequency = <100000>;
		fsl,timeout = <10000>;
	};
samsung-i2c
	i2c@13870000 {
		samsung,i2c-sda-delay = <100>;
		samsung,i2c-max-bus-freq = <100000>;
		samsung,i2c-slave-addr = <0>;
	};
ce4100-i2c.txt
	i2c@0 {
missing clock-frequency
	};
omap-i2c.txt
i2c1: i2c@0 {
    ti,hwmods = "i2c1";   -- I think this is because they are missing reg = <>
    clock-frequency = <400000>;
};
arm-versatile.txt
missing clock frequency
i2c-designware.txt
looks ok
fsl-imx-i2c.txt
looks ok
cpm_qe/cpm/i2c.txt
file is in wrong place

The schema probably should be:
	i2c@address {
		compatible = "xxxx";  - required
		reg = <xxxx>; - required
		interrupts = <xxx>;  - required?
		interrupt-parent = <xxxx>; - if need
		clock-frequency = <xxxx>;  - optional, schema says default is 100Khz
		slave-addr = <xxxx>;  - optional, all of the controllers support slave mode
		timeout = <xxxxx>; - optional
		#address-cells = <1>;
		#size-cells = <0>;
		aaaa,bbbb = <xxxx> - private, must have a comma in the name
      }

Once schemas are in place and everything is validating, another pass
can be made to implement a common parser. The common parser would get
all of these attributes and parse them into a struct.  Then we can
work on making sure each driver actually implements these core
attributes consistently.

-- 
Jon Smirl
jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

Re: Schemas for device trees

[jonsmirl-Re5JQEeQqe8AvxtiuMwx3w]

On Thu, Mar 29, 2012 at 9:59 PM, jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> i2c provides an example of where a schema would be useful.
> From the binding text files, I removed the common items

Another reason for schemas, I just misspelled an attribute. It took me
several hours to figure out why my system wasn't working. I ended up
inserting printks in the driver looking for what was wrong before I
figured out that the attribute was simply misspelled. I used
slave-address when the attribute was slave-addr. Schema would have
flagged the unknown attribute at compile time.

------------------------------------------------------------

Using XML schemas is a crazy idea. It would better to leverage the DTC
and kernel infrastructure. We can use a scheme similar to kernel
module support.

The driver source for each OF compatible device would contain macros
that generate a small schema that describes the device.

Instantiation of the device in a real device tree...
		ads7846@1 {
			compatible = "ti,ads7846";
			spi-max-frequency = <1200000>;
			reg = <1>;
			gpio_pendown = <&gpio_gpio 4 0>;
			gpio_cs = <&gpio_ebi_i2stx_0 3 0>;
		};

schema for this would be something like

{
	compatible = "ti,ads7846";
	inherits-from = "schema,spi-device"
	gpio_pendown* = <&gpio_reference>;  /* star = optional */
	gpio_cs = <&gpio_reference>;
}
for a spi host...
{
	compatible = "nxp,lpc31xx-spi";
	inherits-from = "schema,spi-bus"
};
Compiling these drivers would generate two small files containing the
schema snippets into a temp directory.

Another permanent directory would hold the generic definitions.
{
	compatible = "schema,spi-device";
	inherits-from = "schema,generic-device"
	spi-max-frequency* = <0-50M>; /* range validation */
};
{
	compatible = "schema,generic-device";
	reg = <>;
	interrupts = <>;
};
{
	compatible = "schema,spi-bus";
	inherits-from = "schema,generic-bus"
	contains = {"schema,spi-device"}
};
{
	compatible = "schema,generic-bus";
	reg = <>;
	interrupts = <>;
	#address-cells = <>;
	#size-cells = <>;				
};

Now the DT compiler can read in these schema snippets and use them to
pattern match against the DT being compiled and perform validation.

-- 
Jon Smirl
jonsmirl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org