Re: [XenSummit 2017] Build tools follow up

[Oleksandr Andrushchenko <andr2000@gmail.com>]

ping

On 07/17/2017 03:47 PM, Oleksandr Andrushchenko wrote:
> Hi, all!
>
> This is a follow-up on Xen distribution build systems we saw at the
> summit and invitation for sharing thoughts and ways we build our 
> images and
> distros. I would like to specifically ask OpenXT project to reply with 
> the
> description of their build system so we have clear picture of what is 
> being
> developed and used around. And of course if there are other approaches 
> to do the
> same you are welcome to share those as well.
>
> On our side at EPAM we have developed a distribution which is called 
> xt-distro
> (xt stays for Xen troops [0]) with the following goals in mind:
> 1. Make it possible to easily build different images as a single 
> distribution
>    - Separate images for different domains should be combined into a
>      distribution, e.g. set of images/artifacts required to run them as a
>      system
>    - Xt-distro allows doing so as easy as running bitbake with the 
> predefined
>      target, e.g. “bitbake xt-image”
> 2. Easily deal with different BSPs for different platforms using a 
> unified way
>    with little or no EPAM specific code/scripts
>    - We use bitbake and stripped version of Poky/meta-openembedded, so 
> there are
>      little in-house extensions we added to deal with Yocto based BSPs 
> [1], [2]
>    - We use bitbake’s well defined scripting language [3], so no yet 
> another
>      language to learn
>    - We use Google repo tool to maintain sets of meta layers as 
> manifest files [4]
> 3. Ability to easily collect component revisions, so any build can be
>    reproduced if need be
>    - We save commit IDs of every component of the build including 
> versions of the
>      meta layers
> 4. Ability to easily customize and tune builds, e.g. URIs, 
> versions/commitIDs,
>    branches used
>    - This is purely done in bitbake’s recipe language
> 5. Make patching process easy
>    - With our extensions you can use bblayers.conf, local.conf, 
> patches which
>      are part of a meta layer which is usually a manual step
> 6. Code reuse is a must, e.g. the same set of software must be easily 
> built for
>    different platforms without copying build scripts of the existing 
> components
>    - With bitbake’s meta layers we define generic recipes, e.g. 
> suitable for all
>      platforms [5] (think of it as a library) and tuning meta layers [6]
>      (we use product concept here) which define specific 
> versions/revisions of the
>      components, apply specific patches and add/remove software
> 7. Cross compilation must be an easy task to do
>    - This is easily achieved with Yocto builds and usually not a 
> problem for other
>      build systems as well (with SDKs)
>    - Allows building for x86/ARM and other architectures as well
> 8. There must be a simple way to share build artifacts of different 
> domains between
>    each other, for example, DomU’s kernel should be a part of Dom0’s 
> rootfs, so
>    xl/libxl can access it
>    - This is achieved with bitbake’s recipes, so no EPAM extensions
> 9. Possibility to easily use different build systems for different 
> components of
>    the system.
>    - Bitbake allows you building makefile based projects, CMake etc. 
> out of the box,
>      so adding firmware, stubdoms made easy
> 10. Development support
>    - We use SDKs built by bitbake so during everyday development you 
> are packed with
>      all that is needed to re-create build environment (with both host 
> and target
>      environments, e.g. x86 host tools and ARM cross-compiler)
>    - Speed up builds for developers within organization, e.g. we reuse 
> downloads and
>      build cache between all of the developer’s machines on our network
> 11. Make the build system suitable for build automation
>     - We have created a python script which can easily be used with
>       Jenkins or even cron [7]
>
> For example, with xt-distro our development product delivers: Xen 
> image and
> policy file, Dom0 kernel and dtb images (Dom0 + DomU), Dom0 rootfs 
> image (with
> embedded kernels and configuration for DomU), DomU rootfs image. This
> effectively means, that the build system can provide you with the 
> complete set
> of images to run your product/distribution.
>
> We are looking forward for any kind of feedback and will be glad to 
> collaborate
> on the above.
>
> Thank you,
> Xen-troops at EPAM
>
> [0] https://github.com/xen-troops/xt-distro
> [1] https://git.yoctoproject.org/cgit/cgit.cgi/poky
> [2] http://cgit.openembedded.org/meta-openembedded
> [3] 
> http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html
> [4] https://gerrit.googlesource.com/git-repo
> [5] https://github.com/xen-troops/meta-xt-images
> [6] https://github.com/xen-troops/meta-xt-prod-devel
> [7] https://github.com/xen-troops/build-scripts
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel