[Buildroot] How to handle modularity in buildroot?

[Buildroot] How to handle modularity in buildroot?

[Charles Manning]

Hello All

Ok, first off... I must absolutely congratulate and thank everyone for
a really useful build system.

Until now I had been using openembedded to build smaller file systems
for embedded systems. OE is way overkill for most embedded systems and
is like trying to use a chainsaw to cut butter. Buildroot is much
faster than OE and is way easier to navigate.

There are a few things I could do in OE that I am struggling with in
buildroot - no doubt mainly due to my learner status. One of these is
handling modularity.

Is there an easy way to have modular/hierarchical projects? For
example I would like to have a base file system that holds common
stuff but then have variants (eg. production vs development).

In OE this is easy: just create a "basic system" package which is
included in the recipes for each of the variants.

How would this be tackled in buildroot?

Thanks

Charles

[Buildroot] How to handle modularity in buildroot?

[ANDY KENNEDY]

> -----Original Message-----
> From: buildroot-bounces at busybox.net [mailto:buildroot-bounces at busybox.net] On Behalf Of Charles
> Manning
> Sent: Monday, December 03, 2012 4:01 PM
> To: buildroot at busybox.net
> Subject: [Buildroot] How to handle modularity in buildroot?
> 
> Hello All
> 
> Ok, first off... I must absolutely congratulate and thank everyone for
> a really useful build system.
> 
> Until now I had been using openembedded to build smaller file systems
> for embedded systems. OE is way overkill for most embedded systems and
> is like trying to use a chainsaw to cut butter. Buildroot is much
> faster than OE and is way easier to navigate.
> 
> There are a few things I could do in OE that I am struggling with in
> buildroot - no doubt mainly due to my learner status. One of these is
> handling modularity.
> 
> Is there an easy way to have modular/hierarchical projects? For
> example I would like to have a base file system that holds common
> stuff but then have variants (eg. production vs development).
> 
> In OE this is easy: just create a "basic system" package which is
> included in the recipes for each of the variants.
> 
> How would this be tackled in buildroot?

I believe what you are looking for is 'make defconfig'.  This will 
provide a basic set of configuration options to get you going, then
you'll have to edit the things like processor family, processor type,
whether you want uClibc vs {e,}glibc, etc.

The default config is usually okay for a first attempt, however, I like
to provide to others in my company a set of config files to choose from.
Those, I derive from just cranking through the menuconfig and save the
options off when I get done.

If you are looking for more than this, there are a few other default
config files to attempt.  To list those config files do a 'make help'.

Hope this helps!

Andy

> 
> Thanks
> 
> Charles
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] How to handle modularity in buildroot?

[Charles Manning]

On Tuesday 04 December 2012 12:17:13 ANDY KENNEDY wrote:

> >
> > Is there an easy way to have modular/hierarchical projects? For
> > example I would like to have a base file system that holds common
> > stuff but then have variants (eg. production vs development).
> >
> > In OE this is easy: just create a "basic system" package which is
> > included in the recipes for each of the variants.
> >
> > How would this be tackled in buildroot?
>
> I believe what you are looking for is 'make defconfig'.  This will
> provide a basic set of configuration options to get you going, then
> you'll have to edit the things like processor family, processor type,
> whether you want uClibc vs {e,}glibc, etc.
>
> The default config is usually okay for a first attempt, however, I like
> to provide to others in my company a set of config files to choose from.
> Those, I derive from just cranking through the menuconfig and save the
> options off when I get done.
>
> If you are looking for more than this, there are a few other default
> config files to attempt.  To list those config files do a 'make help'.
>
> Hope this helps!

Thanks Andy

I am aware of this mechanism but I don't think it really gets me what I am 
after.

I think that with defconfig, I would be able to make defconfig, then tweak the 
config to get a variant. But now if I change the base config, I then have to 
go through the process again for the variant.

What I would like is the ability to have:
* A base package
* Production rootfs = base + production files.
* Development rootfs = base  + development files.

Then a change to the base package automatically flows through the production + 
development rootfs.


Is there a way to accomplish that?

Thanks

Charles

[Buildroot] How to handle modularity in buildroot?

[Stephan Hoffmann]

Am 04.12.2012 00:49, schrieb Charles Manning:
> On Tuesday 04 December 2012 12:17:13 ANDY KENNEDY wrote:
>
>>> Is there an easy way to have modular/hierarchical projects? For
>>> example I would like to have a base file system that holds common
>>> stuff but then have variants (eg. production vs development).
>>>
>>> In OE this is easy: just create a "basic system" package which is
>>> included in the recipes for each of the variants.
>>>
>>> How would this be tackled in buildroot?
>> I believe what you are looking for is 'make defconfig'.  This will
>> provide a basic set of configuration options to get you going, then
>> you'll have to edit the things like processor family, processor type,
>> whether you want uClibc vs {e,}glibc, etc.
>>
>> The default config is usually okay for a first attempt, however, I like
>> to provide to others in my company a set of config files to choose from.
>> Those, I derive from just cranking through the menuconfig and save the
>> options off when I get done.
>>
>> If you are looking for more than this, there are a few other default
>> config files to attempt.  To list those config files do a 'make help'.
>>
>> Hope this helps!
> Thanks Andy
>
> I am aware of this mechanism but I don't think it really gets me what I am 
> after.
>
> I think that with defconfig, I would be able to make defconfig, then tweak the 
> config to get a variant. But now if I change the base config, I then have to 
> go through the process again for the variant.
>
> What I would like is the ability to have:
> * A base package
> * Production rootfs = base + production files.
> * Development rootfs = base  + development files.
>
Hi Charles,

I think this could be best done with git. You could create a production
and a development branch both based on your common base. After updating
base both branches could either be rebased or the new base merged in.
> Then a change to the base package automatically flows through the production + 
> development rootfs.
>
Well, not really automatically;-) To be honest, I am currently trying to
find an optimal git workflow for this.

Kind regards

Stephan
> Is there a way to accomplish that?
>
> Thanks
>
> Charles
>
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot


-- 
reLinux     -    Stephan Hoffmann
Am Schmidtgrund 124    50765 K?ln
Tel. +49.221.95595-19    Fax: -64
www.reLinux.de     sho at reLinux.de

[Buildroot] How to handle modularity in buildroot?

[Bjørn Forsman]

On 4 December 2012 00:49, Charles Manning <cdhmanning@gmail.com> wrote:
[...]
> I think that with defconfig, I would be able to make defconfig, then tweak the
> config to get a variant. But now if I change the base config, I then have to
> go through the process again for the variant.
>
> What I would like is the ability to have:
> * A base package
> * Production rootfs = base + production files.
> * Development rootfs = base  + development files.
>
> Then a change to the base package automatically flows through the production +
> development rootfs.
>
>
> Is there a way to accomplish that?

I don't know if it's possible to do that right now, but in principle
this is just a merge of a base config and an addon config. Say, if
there was an include directive in config files you'd be done. Maybe
you could try to create this merged config file by hand and see if
it's something that can be automated later?

1. make a (base) defconfig
2. make N new defconfigs with various extra options and call them addonN_config
3. remove the symbols found in the base config from all addonN_config files

Then, before the build, you do something like "cat base_config
addonN_config > config".

Best regards,
Bj?rn Forsman

[Buildroot] How to handle modularity in buildroot?

[Thomas Petazzoni]

Dear Charles Manning,

On Tue, 4 Dec 2012 12:49:14 +1300, Charles Manning wrote:

> I am aware of this mechanism but I don't think it really gets me what I am 
> after.
> 
> I think that with defconfig, I would be able to make defconfig, then tweak the 
> config to get a variant. But now if I change the base config, I then have to 
> go through the process again for the variant.
> 
> What I would like is the ability to have:
> * A base package
> * Production rootfs = base + production files.
> * Development rootfs = base  + development files.
> 
> Then a change to the base package automatically flows through the production + 
> development rootfs.
> 
> Is there a way to accomplish that?

First of all, thanks for your nice words about Buildroot, definitely
appreciated.

That said, if Buildroot is certainly simpler than OE, it also means
that Buildroot has a smaller feature set, and to some extent a bit less
flexibility.

What you can do to achieve what you want is use defconfig fragments.
For example:

 * mysystem_base_defconfig.frag would contain:

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_PATH="/home/thomas/x-tools/arm-br-arm926/usr/"
BR2_TOOLCHAIN_EXTERNAL_LARGEFILE=y
BR2_TOOLCHAIN_EXTERNAL_INET_IPV6=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_PACKAGE_BUSYBOX=y

 * mysystem_prod_defconfig.frag would contain:

BR2_PACKAGE_APPLICATION1=y
BR2_PACKAGE_APPLICATION2=y

 * mysystem_dev_defconfig.frag would contain:

BR2_PACKAGE_STRACE=y
BR2_PACKAGE_LTRACE=y

And then, when you want to do a production build:

 make clean
 cat mysystem_base_defconfig.frag mysystem_prod_defconfig.frag > configs/mysystem_prod_defconfig
 make mysystem_prod_defconfig
 make

And when you want to do a development build:

 make clean
 cat mysystem_base_defconfig.frag mysystem_dev_defconfig.frag > configs/mysystem_dev_defconfig
 make mysystem_dev_defconfig
 make

Basically, instead of having Buildroot providing this feature, we rely
on the simple manipulation of the configuration file.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] How to handle modularity in buildroot?

[Charles Manning]

On Tue, Dec 4, 2012 at 10:10 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Charles Manning,
>
> On Tue, 4 Dec 2012 12:49:14 +1300, Charles Manning wrote:
>
>> I am aware of this mechanism but I don't think it really gets me what I am
>> after.
>>
>> I think that with defconfig, I would be able to make defconfig, then tweak the
>> config to get a variant. But now if I change the base config, I then have to
>> go through the process again for the variant.
>>
>> What I would like is the ability to have:
>> * A base package
>> * Production rootfs = base + production files.
>> * Development rootfs = base  + development files.
>>
>> Then a change to the base package automatically flows through the production +
>> development rootfs.
>>
>> Is there a way to accomplish that?
>
> First of all, thanks for your nice words about Buildroot, definitely
> appreciated.
>
> That said, if Buildroot is certainly simpler than OE, it also means
> that Buildroot has a smaller feature set, and to some extent a bit less
> flexibility.

Thanks Thomas

I certainly would never want buildroot to bloat out to be like OE. The
most direct way to make something useless is to try to be all o be all
things to all people.

buildroot is very valuable and I intend to contribute where I can. I
live in New Zealand which makes it a bit hard to attend FOSDEM :-).

>
> What you can do to achieve what you want is use defconfig fragments.
> For example:
>
>  * mysystem_base_defconfig.frag would contain:
>
> BR2_arm=y
> BR2_TOOLCHAIN_EXTERNAL=y
> BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> BR2_TOOLCHAIN_EXTERNAL_PATH="/home/thomas/x-tools/arm-br-arm926/usr/"
> BR2_TOOLCHAIN_EXTERNAL_LARGEFILE=y
> BR2_TOOLCHAIN_EXTERNAL_INET_IPV6=y
> BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
> # BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
> BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
> BR2_TOOLCHAIN_EXTERNAL_CXX=y
> BR2_PACKAGE_BUSYBOX=y
>
>  * mysystem_prod_defconfig.frag would contain:
>
> BR2_PACKAGE_APPLICATION1=y
> BR2_PACKAGE_APPLICATION2=y
>
>  * mysystem_dev_defconfig.frag would contain:
>
> BR2_PACKAGE_STRACE=y
> BR2_PACKAGE_LTRACE=y
>
> And then, when you want to do a production build:
>
>  make clean
>  cat mysystem_base_defconfig.frag mysystem_prod_defconfig.frag > configs/mysystem_prod_defconfig
>  make mysystem_prod_defconfig
>  make
>
> And when you want to do a development build:
>
>  make clean
>  cat mysystem_base_defconfig.frag mysystem_dev_defconfig.frag > configs/mysystem_dev_defconfig
>  make mysystem_dev_defconfig
>  make
>
> Basically, instead of having Buildroot providing this feature, we rely
> on the simple manipulation of the configuration file.

This sounds like the right idea.

I will experiment with this a bit and see where it takes me. If there
is something worth sharing I'll do that.

Thanks

Charles

[Buildroot] How to handle modularity in buildroot?

[Arnout Vandecappelle]

On 04/12/12 19:18, Charles Manning wrote:
>> >  And then, when you want to do a production build:
>> >
>> >    make clean
>> >    cat mysystem_base_defconfig.frag mysystem_prod_defconfig.frag>  configs/mysystem_prod_defconfig
>> >    make mysystem_prod_defconfig
>> >    make
>> >
>> >  And when you want to do a development build:
>> >
>> >    make clean
>> >    cat mysystem_base_defconfig.frag mysystem_dev_defconfig.frag>  configs/mysystem_dev_defconfig
>> >    make mysystem_dev_defconfig
>> >    make
>> >
>> >  Basically, instead of having Buildroot providing this feature, we rely
>> >  on the simple manipulation of the configuration file.
> This sounds like the right idea.
>
> I will experiment with this a bit and see where it takes me. If there
> is something worth sharing I'll do that.

  If you don't want to clobber your buildroot tree with
configs/mysystem_dev_defconfig, you can instead do:

  make clean
  cat mysystem_base_defconfig.frag mysystem_dev_defconfig.frag > /tmp/mysystem_dev_defconfig
  make defconfig BR2_DEFCONFIG=/tmp/mysystem_dev_defconfig
  make


  And to create the _dev fragment, you can try something like:

make savedefconfig
sort defconfig mysystem_base_defconfig.frag | uniq -u > mysystem_dev_defconfig.frag

(this is just from the top of my head so YMMV).


  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F