[Buildroot] asking for advice on improving our buildroot setup

[Buildroot] asking for advice on improving our buildroot setup

[Sven Neumann]

Hi,

we are successfully using buildroot for quite a while now. However our
setup is somewhat akward and I'd like to improve it. Now that we have
updated to the latest buildroot I dare to ask for advice here on how you
would implement the following scenario:

We are building factory images, USB rescue images and update images for
our Raumfeld evices. In order to do this we need to build, for each
target platform (2 platforms currently):

 (1) a kernel and initramfs with some basic tools
 (2) an ext2 rootfs that contains factory tests as well
     as tools to format the flash and unpack a tar.gz
     rootfs on it
 (3) the actual rootfs of the devices as tar.gz
     this includes a kernel and modules and this kernel
     may be different from the kernel built in step 1

So what we are doing currently is that we are running full independent
buildroot builds for each of the steps listed above and then build our
images from the results of those runs. That is we pick up the kernel
with initramfs from step 1, combine it with the ext rootfs that is
created in step 2 and add the rootfs.tar.gz from step 3.

Actually step 3 is even performed twice for the ARM platform because we
need two different rootfs for two different hardware types (Controller
and Speaker/Connector).

As you can imagine this build consumes a lot of time and disk space.
Currently I can think of two ways to improve this:

(1) Add an initial step that builds a toolchain and then use that
    toolchain from the following steps. This would save the time
    to rebuild the toolchain for each of the steps.

(2) Combine all steps and build different rootfs from within the
    same buildroot tree. I read that this would be supported. But
    I would like to get advice from you first. Is there a chance
    that this will work for the described setup?


Thanks,
Sven

-- 
Sven Neumann
Head of RAUMFELD Software Development

Lautsprecher Teufel GmbH | B?lowstr. 66 | 10783 Berlin | Germany
Tel: +49 (0)30-300 930 153 | s.neumann at raumfeld.com

[Buildroot] asking for advice on improving our buildroot setup

[Peter Korsgaard]

>>>>> "Sven" == Sven Neumann <s.neumann@raumfeld.com> writes:

Hi,

 Sven> As you can imagine this build consumes a lot of time and disk space.
 Sven> Currently I can think of two ways to improve this:

 Sven> (1) Add an initial step that builds a toolchain and then use that
 Sven>     toolchain from the following steps. This would save the time
 Sven>     to rebuild the toolchain for each of the steps.

I think that is the best/easiest option.

 Sven> (2) Combine all steps and build different rootfs from within the
 Sven>     same buildroot tree. I read that this would be supported. But
 Sven>     I would like to get advice from you first. Is there a chance
 Sven>     that this will work for the described setup?

Unfortunately not, buildroot only knows of a single rootfs - So you'll
need to build the 2 rootfs'es seperately.

The only other option I could think of is perhaps to combine the two
rootfs'es and include the factory test sw in the device rootfs if that's
an option?

-- 
Bye, Peter Korsgaard

[Buildroot] asking for advice on improving our buildroot setup

[Thomas Petazzoni]

Hello Sven,

Thanks for your feedback on this, it is really interesting to see how
people are using Buildroot, what are their use cases, and how we could
improve things to match more use cases.

Le Thu, 07 Jul 2011 12:09:20 +0200,
Sven Neumann <s.neumann@raumfeld.com> a ?crit :

> We are building factory images, USB rescue images and update images
> for our Raumfeld evices. In order to do this we need to build, for
> each target platform (2 platforms currently):
> 
>  (1) a kernel and initramfs with some basic tools
>  (2) an ext2 rootfs that contains factory tests as well
>      as tools to format the flash and unpack a tar.gz
>      rootfs on it
>  (3) the actual rootfs of the devices as tar.gz
>      this includes a kernel and modules and this kernel
>      may be different from the kernel built in step 1
> 
> So what we are doing currently is that we are running full independent
> buildroot builds for each of the steps listed above and then build our
> images from the results of those runs. That is we pick up the kernel
> with initramfs from step 1, combine it with the ext rootfs that is
> created in step 2 and add the rootfs.tar.gz from step 3.

I am not sure to understand why (2) and (3) are separate things here.
Can't you just build a single rootfs with all normal production
applications and factory tests, and after the build simply copy the
rootfs and remove factory tests from that copy ?

> (1) Add an initial step that builds a toolchain and then use that
>     toolchain from the following steps. This would save the time
>     to rebuild the toolchain for each of the steps.

You can even remove that step. Either use a pre-compiled toolchain or
build a toolchain once for all with Crosstool-NG and Buildroot. Keep
that toolchain around as a reference, and use it for all your builds.
To me, it really doesn't make any sense rebuilding a toolchain over and
over again: while it is likely that you make multiple changes to your
kernel and rootfs, it is quite unlikely that you do many changes at the
toolchain level. So just generate it once and use it for all your
builds. Of course, keep the recipe use to generate the toolchain
somewhere, so that if you ever need to regenerate the toolchain to make
some modification, you know how to do it.

> (2) Combine all steps and build different rootfs from within the
>     same buildroot tree. I read that this would be supported. But
>     I would like to get advice from you first. Is there a chance
>     that this will work for the described setup?

There is currently no plan in Buildroot to support more than one rootfs
in one build.

To reduce the build time and number of builds, I would:

 *) Generate the toolchain once for all, as detailed above. If your
 root filesystems are quite small, then there is a big chance that 50,
 60 or 70% of the build time is due to the toolchain building.

 *) Combine your (2) and (3) rootfs into one build, if possible (see my
 question above)

 *) If possible, do not generate two different rootfs for your two
 devices, but generate a single rootfs that works on both (of course
 this is only possible if the CPU architecture is the same)

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] asking for advice on improving our buildroot setup

[Daniel Mack]

On Thu, Jul 7, 2011 at 1:59 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Thu, 07 Jul 2011 12:09:20 +0200,
> Sven Neumann <s.neumann@raumfeld.com> a ?crit :
>
>> We are building factory images, USB rescue images and update images
>> for our Raumfeld evices. In order to do this we need to build, for
>> each target platform (2 platforms currently):
>>
>> ?(1) a kernel and initramfs with some basic tools
>> ?(2) an ext2 rootfs that contains factory tests as well
>> ? ? ?as tools to format the flash and unpack a tar.gz
>> ? ? ?rootfs on it
>> ?(3) the actual rootfs of the devices as tar.gz
>> ? ? ?this includes a kernel and modules and this kernel
>> ? ? ?may be different from the kernel built in step 1
>>
>> So what we are doing currently is that we are running full independent
>> buildroot builds for each of the steps listed above and then build our
>> images from the results of those runs. That is we pick up the kernel
>> with initramfs from step 1, combine it with the ext rootfs that is
>> created in step 2 and add the rootfs.tar.gz from step 3.
>
> I am not sure to understand why (2) and (3) are separate things here.
> Can't you just build a single rootfs with all normal production
> applications and factory tests, and after the build simply copy the
> rootfs and remove factory tests from that copy ?

Let me try to explain the setup a little more, as it's a bit complicated.

These devices are able to boot from a USB-Stick formatted with a FAT
filesystem, and users must be able to create this boot media
themselfes, with whatever OS. They just copy a single file they
downloaded from the website to the stick, press a button on power-up
and the device will recover to it's factory defaults. This is
important for emergency de-bricking and - as it's the easiest solution
- also for factory tests at the production line.

These files start with an uImage that includes a minimal initramfs,
and U-Boot leeches this first part of it to memory and jumps to the
entry point. Unfortunately, the maximum size of that initramfs is
limited, and so we can't squeeze everything in there that we need.

Once this uImage is fully booted from its embedded initramfs, it
mounts the USB-Stick and loop-mounts the same single-file image (with
an offset) and finds another (ext3) filesystem at this offset. This FS
contains a number of scripts for factory testing, UBI tools, other
test files for audio and display tests and, after all, a .tar.gz
archive which contains the rootfs that is extracted to the UBI flash.
Of course, the content of this rootfs.tar.gz is also created by BR2.

We could probably create the initramfs from the final rootfs by
manually deleting tons of files, but that didn't seem more
maintainable after all.


Hope this explains our approach a little better.

Daniel

[Buildroot] asking for advice on improving our buildroot setup

[Arnout Vandecappelle]

 [Late reply, but probably still relevant]

On Thursday 07 July 2011 12:09:20, Sven Neumann wrote:
> As you can imagine this build consumes a lot of time and disk space.
> Currently I can think of two ways to improve this:
> 
> (1) Add an initial step that builds a toolchain and then use that
>     toolchain from the following steps. This would save the time
>     to rebuild the toolchain for each of the steps.
> 
> (2) Combine all steps and build different rootfs from within the
>     same buildroot tree. I read that this would be supported. But
>     I would like to get advice from you first. Is there a chance
>     that this will work for the described setup?


 If your build host can be upgraded to a recent kernel with btrfs, you could 
do the following:

 * Create a basic buildroot configuration with the stuff that is relevant for 
all builds.

 * cp --reflink -a output output-specific-target

 * make specific-target-config O=output-specific-target
   => this will add the additional stuff to output-specific-target and (re) 
create the images.

 Because of the --reflink, no additional disk space will be used for the copy 
of the toolchain etc.


 Regards,
 Arnout

-- 
Arnout Vandecappelle                        Senior Embedded Software Architect
arnout dot vandecappelle at essensium com   +32-16-286540       +32-478-010353
Essensium/Mind                              http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium         BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43