Re: [OE-core] [PATCH 2/5] meta/files: add layer setup JSON schema and example

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Wed, 2022-08-17 at 15:10 +0200, Alexander Kanavin wrote:
> From: Joshua Watt <JPEWhacker@gmail.com>
> 
> Defines a common schema for layer setup that can be consumed by tools to
> know how to fetch and assemble layers for end users. Also includes an
> example of the layer setup that constructs poky/meta-intel/imaginary product layer
> for reference.
> 
> The schema can be used to validate a layer setup file with the commands:
> 
>  $ python3 -m pip install jsonschema
>  $ jsonschema -i meta/files/layers.example.json meta/files/layers.schema.json
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> 
> Alex: I made the following modifications to Joshua's original commit:
> 
> - moved the files from meta/lib to meta/files
> 
> - the example json showcases a multi-repo, multi-layer setup with additional
> configurations and machines, instead of just poky - closer to a typical product
> 
> - added oe-selftest that validates the example json against the schema using python3-jsonschema-native
> 
> - the schema is modified so that:
> 
> -- all lists (sources, layers, remotes) are replaced by objects keyed by 'name' properties of the list items.
> This allows using them as dicts inside Python, and makes the json more compact and readable.
> 
> -- added 'contains_this_file' property to source object
> 
> -- added 'buildconfigs', 'machines' and 'distros' properties to layer objects.
> 
> -- replaced 'remote' property with a 'oneOf' definition for git with a specific
> 'git-remote' property. 'oneOf' is problematic when schema validation fails:
> the diagnostic is only that none of oneOf variants matched, which is too non-specific.
> 
> -- added 'describe' property to 'git-remote' object.
> 
> -- removed description property for a layer source: it is not clear how to add that
> when auto-generating the json

I finally got some time to think about this a little bit more.

One thing that concerns me a little is that this mixes two things,
layer setup (as in repos) and configuration. I'm nervous about json
config which effectively has to duplicate the list of machines/distros
in layers.

Where is the distro/machine data used?

Are these the only config options people will want to add or will we
have others? init system? libc? Or feature information about which
configurations are expected to work? I worry this is a magnet for
future feature creep and duplication of information.

I can see where the buildconfigs are used but again, does the json file
need to encode those or could you determine them by inspection of the
layers once setup?

I'm still worried about bolting a format directly into one of our core
tools as it effectively means it is a definitive standard. We do
probably need one but I'm not convinced this is quite right, perhaps
because of the above reason. Not 100% sure but something doesn't feel
quite right.

Also, if there is new version of this series, could you squash in these
copyright/license tweaks please:

https://git.yoctoproject.org/poky/commit/?h=master-next&id=45b396298c1dd638bb702f5251b4a663f07978ca

Cheers,

Richard