How to check (an updated) schema if make dt_binding_check fails already?

How to check (an updated) schema if make dt_binding_check fails already?

[Andy Shevchenko]

Hi!

I would like to extend an existing schema, the checker currently (on Debian
unstable) fails with the recent in-kernel schema. What should I do?

$ make dt_binding_check DT_SCHEMA_FILES=/usb/snps,dwc3.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.json
  Traceback (most recent call last):
    File "/usr/bin/dt-mk-schema", line 8, in <module>
        sys.exit(main())
                 ^^^^^^
      File "/usr/lib/python3/dist-packages/dtschema/mk_schema.py", line 28, in main
        schemas = dtschema.DTValidator(args.schemas).schemas
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 363, in __init__
        self.make_property_type_cache()
      File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 420, in make_property_type_cache
        self.props, self.pat_props = get_prop_types(self.schemas)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 187, in get_prop_types
        del props[r'^[a-z][a-z0-9\-]*$']
            ~~~~~^^^^^^^^^^^^^^^^^^^^^^^
    KeyError: '^[a-z][a-z0-9\\-]*$'
    make[2]: *** [Documentation/devicetree/bindings/Makefile:63: Documentation/devicetree/bindings/processed-schema.json] Error 1
    make[2]: *** Deleting file 'Documentation/devicetree/bindings/processed-schema.json'
    make[1]: *** Makefile:1509: dt_binding_schemas] Error 2
    make: *** [Makefile:251: __sub-make] Error 2

Tree: Linux Next next-20241212

DT schema:
$ apt list dt-schema*
dt-schema/unstable,unstable,now 2023.11-3 all [installed]

Library:
$ apt list python3-libfdt*
python3-libfdt/testing,testing,unstable,now 1.7.0-2+b2 amd64 [installed]

-- 
With Best Regards,
Andy Shevchenko

Re: How to check (an updated) schema if make dt_binding_check fails already?

[Andy Shevchenko]

On Thu, Dec 12, 2024 at 08:00:47PM +0200, Andy Shevchenko wrote:
> 
> Hi!
> 
> I would like to extend an existing schema, the checker currently (on Debian
> unstable) fails with the recent in-kernel schema. What should I do?
> 
> $ make dt_binding_check DT_SCHEMA_FILES=/usb/snps,dwc3.yaml
>   SCHEMA  Documentation/devicetree/bindings/processed-schema.json
>   Traceback (most recent call last):
>     File "/usr/bin/dt-mk-schema", line 8, in <module>
>         sys.exit(main())
>                  ^^^^^^
>       File "/usr/lib/python3/dist-packages/dtschema/mk_schema.py", line 28, in main
>         schemas = dtschema.DTValidator(args.schemas).schemas
>                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>       File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 363, in __init__
>         self.make_property_type_cache()
>       File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 420, in make_property_type_cache
>         self.props, self.pat_props = get_prop_types(self.schemas)
>                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>       File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 187, in get_prop_types
>         del props[r'^[a-z][a-z0-9\-]*$']
>             ~~~~~^^^^^^^^^^^^^^^^^^^^^^^
>     KeyError: '^[a-z][a-z0-9\\-]*$'
>     make[2]: *** [Documentation/devicetree/bindings/Makefile:63: Documentation/devicetree/bindings/processed-schema.json] Error 1
>     make[2]: *** Deleting file 'Documentation/devicetree/bindings/processed-schema.json'
>     make[1]: *** Makefile:1509: dt_binding_schemas] Error 2
>     make: *** [Makefile:251: __sub-make] Error 2

FWIW, this traceback happens independently on presence or content of
the DT_SCHEMA_FILE variable.

So, any suggestions, please? Can this be fixed rather sooner than later?

> Tree: Linux Next next-20241212
> 
> DT schema:
> $ apt list dt-schema*
> dt-schema/unstable,unstable,now 2023.11-3 all [installed]
> 
> Library:
> $ apt list python3-libfdt*
> python3-libfdt/testing,testing,unstable,now 1.7.0-2+b2 amd64 [installed]

-- 
With Best Regards,
Andy Shevchenko

Re: How to check (an updated) schema if make dt_binding_check fails already?

[Krzysztof Kozlowski]

On 12/12/2024 19:00, Andy Shevchenko wrote:
> 
> Hi!
> 
> I would like to extend an existing schema, the checker currently (on Debian
> unstable) fails with the recent in-kernel schema. What should I do?


> dt-schema/unstable,unstable,now 2023.11-3 all [installed]

That's some very, very old schema. I don't think such packages are well
maintained in distros. Please use current latest release, through pip or
pipx. See the docs (writing schema), although pip3 is a bit outdated and
on newer machines this is pipx.

> 
> Library:
> $ apt list python3-libfdt*
> python3-libfdt/testing,testing,unstable,now 1.7.0-2+b2 amd64 [installed]
> 


Best regards,
Krzysztof

Re: How to check (an updated) schema if make dt_binding_check fails already?

[Krzysztof Kozlowski]

On 13/12/2024 06:56, Andy Shevchenko wrote:
> On Thu, Dec 12, 2024 at 08:00:47PM +0200, Andy Shevchenko wrote:
>>
>> Hi!
>>
>> I would like to extend an existing schema, the checker currently (on Debian
>> unstable) fails with the recent in-kernel schema. What should I do?
>>
>> $ make dt_binding_check DT_SCHEMA_FILES=/usb/snps,dwc3.yaml
>>   SCHEMA  Documentation/devicetree/bindings/processed-schema.json
>>   Traceback (most recent call last):
>>     File "/usr/bin/dt-mk-schema", line 8, in <module>
>>         sys.exit(main())
>>                  ^^^^^^
>>       File "/usr/lib/python3/dist-packages/dtschema/mk_schema.py", line 28, in main
>>         schemas = dtschema.DTValidator(args.schemas).schemas
>>                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>       File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 363, in __init__
>>         self.make_property_type_cache()
>>       File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 420, in make_property_type_cache
>>         self.props, self.pat_props = get_prop_types(self.schemas)
>>                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>       File "/usr/lib/python3/dist-packages/dtschema/validator.py", line 187, in get_prop_types
>>         del props[r'^[a-z][a-z0-9\-]*$']
>>             ~~~~~^^^^^^^^^^^^^^^^^^^^^^^
>>     KeyError: '^[a-z][a-z0-9\\-]*$'
>>     make[2]: *** [Documentation/devicetree/bindings/Makefile:63: Documentation/devicetree/bindings/processed-schema.json] Error 1
>>     make[2]: *** Deleting file 'Documentation/devicetree/bindings/processed-schema.json'
>>     make[1]: *** Makefile:1509: dt_binding_schemas] Error 2
>>     make: *** [Makefile:251: __sub-make] Error 2
> 
> FWIW, this traceback happens independently on presence or content of
> the DT_SCHEMA_FILE variable.
> 
> So, any suggestions, please? Can this be fixed rather sooner than later?
dtschema is kind of developers tool, still being actively changing, not
a stable product, so we expect people running moderately recent
releases. Recent could mean, one or two months old. Not year old, that's
like ancient world for us. Sorry.

And it is not only because of changes in the tool itself, but also
because dtschema contains the core schema. If you use something from
2023, you miss entire new schema and new bindings. There is no way
recent kernel will pass such checks because of that missing bindings,
regardless of actual tool issues like above.

We all use and *only* recommend pip (or pipx mentioned earlier for some
distros).

Best regards,
Krzysztof