RFC: Layer tooling brainstorming

[Richard Purdie <richard.purdie@linuxfoundation.org>]

Layer Tooling for OpenEmbedded
==============================

As we move forward with encouraging the use of metadata layers, a number
of issues are appearing which imply the need for either infrastructure
improvements to the layer construction, or better tools to handle the
layers. The OS TSC discussed layers and layer tooling at the face to
face TSC meeting in San Francisco. Some example problems that were
identified that currently need addressing in some form are:

* There is no dependency information between layers (layer X depends on 
  layer Y and Z)
* There is no version information present (layer X needs version A of 
  layer Y)
* There is no layer "ABI" version number present in layers
* Layer specific sanity checks? (other types of sanity checks on Yocto 
  1.1 roadmap)
* Ability to fetch layers from other repositories
  - Multi SCM?
  - Use bitbake fetcher?
* Ability to generate repositories representing composited layers (Poky,
  RP has hacky scripts)
* Tool to seperate patch into one for different composited parts
* Ability to include partial components of repositories (e.g. only 
  beagleboard from meta-ti)
* Tool to merge (flatten) several layers into one
* Able to visualise dependencies between layers
* When bitbake prints banner of branch/commit, need to cover all layer 
  components present
* bitbake -e on steroids to highlight differences due to layer changes? 
  (long term goal)

Implementation thoughts:

* This layer tooling probably belongs at a higher level on the stack 
  than bitbake itself
* Maybe need to split into "bootstrap" steps (e.g where pseduo is 
  established, layers downloaded etc)
* Need some kind of config file defining layer's properties or at least 
  enhance layer.conf significantly
* git submodules and repo should be looked at but due to specific needs 
  and ability to control tool evolution, likely need own tool.

This hopefully summarises the ideas we discussed in person, we therefore
need to start looking at how to move forward from here with this but I
wanted to document this for starters!

Cheers,

Richard