viability of dt-schema long-term

viability of dt-schema long-term

[mturney]

Folks,
I am advocating use of dt-schema project internally to validate DTS 
files.
I should add that our use is outside kernel tree on proprietary project.

One of the push-backs I'm getting from the management chain is along the 
lines of...

Who is this Rob Herring guy and why should we use a project that is only 
sourced on https://github.com/robherring/dt-schema?
If the kernel project is using it, why isn't kernel.org hosting the 
project?
What is kernel plan if Rob walks away from the project, is this going to 
wither away and die?

There are more, but the above pseudo-quotes grab the gist of the 
management complaints.

Q.1) Is there a plan for the kernel project to suck dt-schema into its 
orbit?

Q.2) How many active maintainers are there for dt-schema?

Q.3) How do I respond to the above types of complaints?

Cheers,
T.mike

Re: viability of dt-schema long-term

[Rob Herring]

On Wed, Nov 25, 2020 at 2:13 PM <mturney@codeaurora.org> wrote:
>
>
> Folks,
> I am advocating use of dt-schema project internally to validate DTS
> files.
> I should add that our use is outside kernel tree on proprietary project.
>
> One of the push-backs I'm getting from the management chain is along the
> lines of...
>
> Who is this Rob Herring guy and why should we use a project that is only
> sourced on https://github.com/robherring/dt-schema?

I wouldn't trust him...

That's the wrong repo though: https://github.com/devicetree-org/dt-schema

(Unfortunately, GH's forks is misleading as the 'root' repo has changed.)

> If the kernel project is using it, why isn't kernel.org hosting the
> project?

It's not a kernel project. That's why devicetree.org hosts it.

> What is kernel plan if Rob walks away from the project, is this going to
> wither away and die?

IMO, only if folks don't find validation valuable or a better
implementation comes along.

> There are more, but the above pseudo-quotes grab the gist of the
> management complaints.
>
> Q.1) Is there a plan for the kernel project to suck dt-schema into its
> orbit?

No, the 'plan' (more like desire) is more in the opposite direction.
Move more of DT (bindings and dts files) out of the kernel for other
projects to use. For now, we have the 'devicetree-rebasing' tree which
is just the DT bits from the kernel tree.

> Q.2) How many active maintainers are there for dt-schema?

Mostly just me. Maxime Ripard is also one. Others could be if the need arose.

> Q.3) How do I respond to the above types of complaints?

jsonschema python module which is our main dependency is also just a
single maintainer. So is dtc. Maybe not what you want to highlight.

Rob

Re: viability of dt-schema long-term

[mturney]

[sending to you directly because I am no longer on kernel list and you 
respond to the dtschema queries anyway, if you prefer I will re-join 
kernel list and re-send to whole list]

Reviving this thread now that we have some experience with dt-schema.

Our experience with both dtc and the dt-schema scripts is there doesn't 
seem to be any real distinction between errors and warnings.
Below are some examples.

This is from dt-validate: : pinctrl@f100000: 'width' is a required 
property
This is from dtc: : Warning (reg_format): /soc/pinctrl@f100000:reg: 
property has invalid length (8 bytes) (#address-cells == 2, #size-cells 
== 1)

In both cases neither tool returned an error code to the shell (echo $? 
= 0)
dtc will error with a syntax problem, but that seems to be it.

Is this how the kernel community prefers these tools to work?

Our concern is more with the dtschema scripts so we can use this to 
break the build and force the engineer to fix either the .yaml or .dtsi 
file.

Before we dive into the dtschema scripts we wanted to understand the 
philosophy behind the design decisions.


On 2020-11-30 11:02, Rob Herring wrote:
> On Wed, Nov 25, 2020 at 2:13 PM <mturney@codeaurora.org> wrote:
>> 
>> 
>> Folks,
>> I am advocating use of dt-schema project internally to validate DTS
>> files.
>> I should add that our use is outside kernel tree on proprietary 
>> project.
>> 
>> One of the push-backs I'm getting from the management chain is along 
>> the
>> lines of...
>> 
>> Who is this Rob Herring guy and why should we use a project that is 
>> only
>> sourced on https://github.com/robherring/dt-schema?
> 
> I wouldn't trust him...
> 
> That's the wrong repo though: 
> https://github.com/devicetree-org/dt-schema
> 
> (Unfortunately, GH's forks is misleading as the 'root' repo has 
> changed.)
> 
>> If the kernel project is using it, why isn't kernel.org hosting the
>> project?
> 
> It's not a kernel project. That's why devicetree.org hosts it.
> 
>> What is kernel plan if Rob walks away from the project, is this going 
>> to
>> wither away and die?
> 
> IMO, only if folks don't find validation valuable or a better
> implementation comes along.
> 
>> There are more, but the above pseudo-quotes grab the gist of the
>> management complaints.
>> 
>> Q.1) Is there a plan for the kernel project to suck dt-schema into its
>> orbit?
> 
> No, the 'plan' (more like desire) is more in the opposite direction.
> Move more of DT (bindings and dts files) out of the kernel for other
> projects to use. For now, we have the 'devicetree-rebasing' tree which
> is just the DT bits from the kernel tree.
> 
>> Q.2) How many active maintainers are there for dt-schema?
> 
> Mostly just me. Maxime Ripard is also one. Others could be if the need 
> arose.
> 
>> Q.3) How do I respond to the above types of complaints?
> 
> jsonschema python module which is our main dependency is also just a
> single maintainer. So is dtc. Maybe not what you want to highlight.
> 
> Rob

Re: viability of dt-schema long-term

[Rob Herring]

On Wed, Sep 29, 2021 at 7:08 PM <mturney@codeaurora.org> wrote:
>
> [sending to you directly because I am no longer on kernel list and you
> respond to the dtschema queries anyway, if you prefer I will re-join
> kernel list and re-send to whole list]
>
> Reviving this thread now that we have some experience with dt-schema.
>
> Our experience with both dtc and the dt-schema scripts is there doesn't
> seem to be any real distinction between errors and warnings.

Everything is an error, but there are too many (for dtbs) or
introduced too frequently (for bindings), so they are all warnings.

What I really want to distinguish is warnings/errors for new
bindings/users vs. existing ones. Things we already have and can't
change, but don't want new ones.

> Below are some examples.
>
> This is from dt-validate: : pinctrl@f100000: 'width' is a required
> property
> This is from dtc: : Warning (reg_format): /soc/pinctrl@f100000:reg:
> property has invalid length (8 bytes) (#address-cells == 2, #size-cells
> == 1)
>
> In both cases neither tool returned an error code to the shell (echo $?
> = 0)
> dtc will error with a syntax problem, but that seems to be it.
>
> Is this how the kernel community prefers these tools to work?

The standard for the kernel is no warnings in general (perhaps you saw
the recent -Werror change). So it's really only a question of erroring
out or not.

Both submitters and maintainers frequently don't run the schema checks
and things get applied. It's better than it was, but still happens. If
we errored out, those that do run checks would always have a broken
tree. In fact, we did error out in some cases and I made it not. I
always run 'make -k', but it seems many don't and miss testing their
stuff due to erroring out.

dtc could probably be changed to error out at least for 'dtbs' target.
Though it would need a -Werror type option or you'd have to list every
warning option on the command line.

> Our concern is more with the dtschema scripts so we can use this to
> break the build and force the engineer to fix either the .yaml or .dtsi
> file.
>
> Before we dive into the dtschema scripts we wanted to understand the
> philosophy behind the design decisions.
>
>
> On 2020-11-30 11:02, Rob Herring wrote:
> > On Wed, Nov 25, 2020 at 2:13 PM <mturney@codeaurora.org> wrote:
> >>
> >>
> >> Folks,
> >> I am advocating use of dt-schema project internally to validate DTS
> >> files.
> >> I should add that our use is outside kernel tree on proprietary
> >> project.
> >>
> >> One of the push-backs I'm getting from the management chain is along
> >> the
> >> lines of...
> >>
> >> Who is this Rob Herring guy and why should we use a project that is
> >> only
> >> sourced on https://github.com/robherring/dt-schema?
> >
> > I wouldn't trust him...
> >
> > That's the wrong repo though:
> > https://github.com/devicetree-org/dt-schema
> >
> > (Unfortunately, GH's forks is misleading as the 'root' repo has
> > changed.)
> >
> >> If the kernel project is using it, why isn't kernel.org hosting the
> >> project?
> >
> > It's not a kernel project. That's why devicetree.org hosts it.
> >
> >> What is kernel plan if Rob walks away from the project, is this going
> >> to
> >> wither away and die?
> >
> > IMO, only if folks don't find validation valuable or a better
> > implementation comes along.
> >
> >> There are more, but the above pseudo-quotes grab the gist of the
> >> management complaints.
> >>
> >> Q.1) Is there a plan for the kernel project to suck dt-schema into its
> >> orbit?
> >
> > No, the 'plan' (more like desire) is more in the opposite direction.
> > Move more of DT (bindings and dts files) out of the kernel for other
> > projects to use. For now, we have the 'devicetree-rebasing' tree which
> > is just the DT bits from the kernel tree.
> >
> >> Q.2) How many active maintainers are there for dt-schema?
> >
> > Mostly just me. Maxime Ripard is also one. Others could be if the need
> > arose.
> >
> >> Q.3) How do I respond to the above types of complaints?
> >
> > jsonschema python module which is our main dependency is also just a
> > single maintainer. So is dtc. Maybe not what you want to highlight.
> >
> > Rob