[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Thomas Petazzoni]

Hello,

I would like to propose a few patches to start a cleanup of
Buildroot. My goal is not only to cleanup Buildroot internally, but
also to clean it up so that it is easier to understand for our users.

This thread is meant to be a preliminary discussion only and is not
yet a request for inclusion, because :

 * Some of the proposed changes remove features that I think confuse
   our users and are not central in Buildroot operation, but others
   might have different opinion on the matter ;

 * The changes are relatively invasive and I did only limited testing
   on them.

 * The documentation is not updated with the changes. I'm waiting for
   the community opinion on the changes before spending time to update
   the documentation

While the commit logs gives all the details, I would like to give an
quick overview of the proposed changes :

 1. Remove the "project" feature.

    This feature is meant to allow an user to build a Linux system for
    several similar platforms where the differences are only in
    Busybox, kernel and U-Boot configuration. While this feature might
    be useful in some very specific cases, my opinion is that it adds
    far too much complexity in Buildroot compared to the added-value
    it gives.

    Removing this feature allows to instantly make the directory
    layout a lot simpler to understand for a newcomer (no strane
    project_build_ARCH directory anymore !)

 2. Remove the BOARD/LOCAL feature.

    This mechanism is a duplication of something that could already be
    done in a different way: by adding a new target in target/device/.

 3. By default, put the output directories in a single directory,
    named output/.

    Nothing complex, it just clarifies the tree for a new user.

 4. Remove BR2_TOPDIR_PREFIX and BR2_TOPDIR_SUFFIX

    The same thing can be done with O=

 5. Rename the output directories with more sensible names.

    In the output directory, we now have

     - build/    where all the packages are built
     - images/   where the final kernel and rootfs images are stored
     - staging/  the staging directory (containing the development files
                 and libraries compiled for the target)
     - target/   which contains the target root filesystem
     - host/     which contains all the host programs
     - stamps/   which contains the stamps files

Thanks for your input,

Thomas

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Will Newton]

On Mon, Sep 7, 2009 at 11:09 PM, Thomas
Petazzoni<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> I would like to propose a few patches to start a cleanup of
> Buildroot. My goal is not only to cleanup Buildroot internally, but
> also to clean it up so that it is easier to understand for our users.
>
> This thread is meant to be a preliminary discussion only and is not
> yet a request for inclusion, because :
>
> ?* Some of the proposed changes remove features that I think confuse
> ? our users and are not central in Buildroot operation, but others
> ? might have different opinion on the matter ;
>
> ?* The changes are relatively invasive and I did only limited testing
> ? on them.
>
> ?* The documentation is not updated with the changes. I'm waiting for
> ? the community opinion on the changes before spending time to update
> ? the documentation
>
> While the commit logs gives all the details, I would like to give an
> quick overview of the proposed changes :
>
> ?1. Remove the "project" feature.
>
> ? ?This feature is meant to allow an user to build a Linux system for
> ? ?several similar platforms where the differences are only in
> ? ?Busybox, kernel and U-Boot configuration. While this feature might
> ? ?be useful in some very specific cases, my opinion is that it adds
> ? ?far too much complexity in Buildroot compared to the added-value
> ? ?it gives.
>
> ? ?Removing this feature allows to instantly make the directory
> ? ?layout a lot simpler to understand for a newcomer (no strane
> ? ?project_build_ARCH directory anymore !)

I think this is probably the right thing to do, the project support
has never made much sense to me.

> ?2. Remove the BOARD/LOCAL feature.
>
> ? ?This mechanism is a duplication of something that could already be
> ? ?done in a different way: by adding a new target in target/device/.

I didn't know this existed.

> ?3. By default, put the output directories in a single directory,
> ? ?named output/.
>
> ? ?Nothing complex, it just clarifies the tree for a new user.
>
> ?4. Remove BR2_TOPDIR_PREFIX and BR2_TOPDIR_SUFFIX
>
> ? ?The same thing can be done with O=

I didn't know this existed either!

> ?5. Rename the output directories with more sensible names.
>
> ? ?In the output directory, we now have
>
> ? ? - build/ ? ?where all the packages are built
> ? ? - images/ ? where the final kernel and rootfs images are stored
> ? ? - staging/ ?the staging directory (containing the development files
> ? ? ? ? ? ? ? ? and libraries compiled for the target)
> ? ? - target/ ? which contains the target root filesystem
> ? ? - host/ ? ? which contains all the host programs
> ? ? - stamps/ ? which contains the stamps files
>
> Thanks for your input,

One thing I want to do quite often is to keep my toolchain but rebuild
my target fs from scratch, so:

# rm -rf build_i686 project_build_i686
# make

It looks like this might be a little bit more complex now. It would be
good to get some makefile targets that can do this, it's possible they
already exist but I'm just not aware of them.

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Jonathan dumaresq]

-----Message d'origine-----
De?: buildroot-bounces at busybox.net [mailto:buildroot-bounces at busybox.net] De
la part de Will Newton
Envoy??: 8 septembre 2009 13:04
??: Thomas Petazzoni
Cc?: buildroot at uclibc.org
Objet?: Re: [Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

On Mon, Sep 7, 2009 at 11:09 PM, Thomas
Petazzoni<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> I would like to propose a few patches to start a cleanup of
> Buildroot. My goal is not only to cleanup Buildroot internally, but
> also to clean it up so that it is easier to understand for our users.
>
> This thread is meant to be a preliminary discussion only and is not
> yet a request for inclusion, because :
>
> ?* Some of the proposed changes remove features that I think confuse
> ? our users and are not central in Buildroot operation, but others
> ? might have different opinion on the matter ;
>
> ?* The changes are relatively invasive and I did only limited testing
> ? on them.
>
> ?* The documentation is not updated with the changes. I'm waiting for
> ? the community opinion on the changes before spending time to update
> ? the documentation
>
> While the commit logs gives all the details, I would like to give an
> quick overview of the proposed changes :
>
> ?1. Remove the "project" feature.
>
> ? ?This feature is meant to allow an user to build a Linux system for
> ? ?several similar platforms where the differences are only in
> ? ?Busybox, kernel and U-Boot configuration. While this feature might
> ? ?be useful in some very specific cases, my opinion is that it adds
> ? ?far too much complexity in Buildroot compared to the added-value
> ? ?it gives.
>
> ? ?Removing this feature allows to instantly make the directory
> ? ?layout a lot simpler to understand for a newcomer (no strane
> ? ?project_build_ARCH directory anymore !)

I think this is probably the right thing to do, the project support
has never made much sense to me.

> ?2. Remove the BOARD/LOCAL feature.
>
> ? ?This mechanism is a duplication of something that could already be
> ? ?done in a different way: by adding a new target in target/device/.

I didn't know this existed.

> ?3. By default, put the output directories in a single directory,
> ? ?named output/.
>
> ? ?Nothing complex, it just clarifies the tree for a new user.
>
> ?4. Remove BR2_TOPDIR_PREFIX and BR2_TOPDIR_SUFFIX
>
> ? ?The same thing can be done with O=

I didn't know this existed either!

> ?5. Rename the output directories with more sensible names.
>
> ? ?In the output directory, we now have
>
> ? ? - build/ ? ?where all the packages are built
> ? ? - images/ ? where the final kernel and rootfs images are stored
> ? ? - staging/ ?the staging directory (containing the development files
> ? ? ? ? ? ? ? ? and libraries compiled for the target)
> ? ? - target/ ? which contains the target root filesystem
> ? ? - host/ ? ? which contains all the host programs
> ? ? - stamps/ ? which contains the stamps files
>
> Thanks for your input,

One thing I want to do quite often is to keep my toolchain but rebuild
my target fs from scratch, so:

# rm -rf build_i686 project_build_i686
# make

It looks like this might be a little bit more complex now. It would be
good to get some makefile targets that can do this, it's possible they
already exist but I'm just not aware of them.

I'm in same position. This could be great !

Jonathan
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[angus salkeld]

Comments inline.

On Tue, 2009-09-08 at 00:09 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> I would like to propose a few patches to start a cleanup of
> Buildroot. My goal is not only to cleanup Buildroot internally, but
> also to clean it up so that it is easier to understand for our users.
> 
> This thread is meant to be a preliminary discussion only and is not
> yet a request for inclusion, because :
> 
>  * Some of the proposed changes remove features that I think confuse
>    our users and are not central in Buildroot operation, but others
>    might have different opinion on the matter ;
> 
>  * The changes are relatively invasive and I did only limited testing
>    on them.
> 
>  * The documentation is not updated with the changes. I'm waiting for
>    the community opinion on the changes before spending time to update
>    the documentation
> 
> While the commit logs gives all the details, I would like to give an
> quick overview of the proposed changes :
> 
>  1. Remove the "project" feature.
> 
>     This feature is meant to allow an user to build a Linux system for
>     several similar platforms where the differences are only in
>     Busybox, kernel and U-Boot configuration. While this feature might
>     be useful in some very specific cases, my opinion is that it adds
>     far too much complexity in Buildroot compared to the added-value
>     it gives.
> 
>     Removing this feature allows to instantly make the directory
>     layout a lot simpler to understand for a newcomer (no strane
>     project_build_ARCH directory anymore !)
> 
>  2. Remove the BOARD/LOCAL feature.
> 
>     This mechanism is a duplication of something that could already be
>     done in a different way: by adding a new target in target/device/.
> 
Quite a few of our engineers like this feature.
When you are testing that a change doesn't break other targets it is
nice to have a shell per "board" and just be able to run "make" in
each to re-build.

Is the O= a direct replacement for this?


>  3. By default, put the output directories in a single directory,
>     named output/.
> 
>     Nothing complex, it just clarifies the tree for a new user.
> 
>  4. Remove BR2_TOPDIR_PREFIX and BR2_TOPDIR_SUFFIX
> 
>     The same thing can be done with O=
> 
>  5. Rename the output directories with more sensible names.
> 
>     In the output directory, we now have
> 
>      - build/    where all the packages are built
>      - images/   where the final kernel and rootfs images are stored
>      - staging/  the staging directory (containing the development files
>                  and libraries compiled for the target)
>      - target/   which contains the target root filesystem
>      - host/     which contains all the host programs
>      - stamps/   which contains the stamps files
> 

We have a a number of boards to build and this is going put the burden
on the end user to handle moving the output directory around.
Not very user friendly for people who use buildroot in anything but the
simplest case.

Please still support the ability to build multiple targets within the
same buildroot (in configuration). One option would be to make "O=<dir>"
a configuration option (defaulting to the output directory).


Another suggestion would be to completely separate out the toolchain
into a new package "buildcross.git" that only builds the cross
toolchain and then buildroot always uses the external toolchain.
It would make buildroot's toolchain setup simpler (one toolchain mode).
And make sharing toolchains more obvious.

Regards
Angus Salkeld

> Thanks for your input,
> 
> Thomas
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Sven Neumann]

Hi,

On Wed, 2009-09-09 at 07:54 +1200, angus salkeld wrote:

> Another suggestion would be to completely separate out the toolchain
> into a new package "buildcross.git" that only builds the cross
> toolchain and then buildroot always uses the external toolchain.
> It would make buildroot's toolchain setup simpler (one toolchain mode).
> And make sharing toolchains more obvious.

I like that idea. Can't really judge if it would be a success, but it
seems like a nice solution that has the potential to simplify things.


Sven

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Peter Korsgaard]

>>>>> "Will" == Will Newton <will.newton@gmail.com> writes:

Hi,

 >> ?4. Remove BR2_TOPDIR_PREFIX and BR2_TOPDIR_SUFFIX
 >> 
 >> ? ?The same thing can be done with O=

 Will> I didn't know this existed either!

It's documented though:

http://buildroot.net/buildroot.html#building_out_of_tree

-- 
Bye, Peter Korsgaard

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Peter Korsgaard]

>>>>> "Sven" == Sven Neumann <s.neumann@raumfeld.com> writes:

 Sven> Hi,
 Sven> On Wed, 2009-09-09 at 07:54 +1200, angus salkeld wrote:

 >> Another suggestion would be to completely separate out the toolchain
 >> into a new package "buildcross.git" that only builds the cross
 >> toolchain and then buildroot always uses the external toolchain.
 >> It would make buildroot's toolchain setup simpler (one toolchain mode).
 >> And make sharing toolchains more obvious.

 Sven> I like that idea. Can't really judge if it would be a success, but it
 Sven> seems like a nice solution that has the potential to simplify things.

I have been thinking about it myself. *REALLY* supporting toolchains for
all the different architectures / variants, uclibc and (e)glibc and
following up on compiler bugs takes quite some effort (and noone seems
willing to do it for buildroot), so maybe the best long term solution is
simply to deprecate the toolchain building stuff and just recommend
crosstool-ng instead?

This would make BR less selfcontained / one-stop-shop though.

-- 
Bye, Peter Korsgaard

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[angus salkeld]

>  Sven> Hi,
>  Sven> On Wed, 2009-09-09 at 07:54 +1200, angus salkeld wrote:
> 
>  >> Another suggestion would be to completely separate out the toolchain
>  >> into a new package "buildcross.git" that only builds the cross
>  >> toolchain and then buildroot always uses the external toolchain.
>  >> It would make buildroot's toolchain setup simpler (one toolchain mode).
>  >> And make sharing toolchains more obvious.
> 
>  Sven> I like that idea. Can't really judge if it would be a success, but it
>  Sven> seems like a nice solution that has the potential to simplify things.
> 
> I have been thinking about it myself. *REALLY* supporting toolchains for
> all the different architectures / variants, uclibc and (e)glibc and
> following up on compiler bugs takes quite some effort (and noone seems
> willing to do it for buildroot), so maybe the best long term solution is
> simply to deprecate the toolchain building stuff and just recommend
> crosstool-ng instead?
> 
> This would make BR less selfcontained / one-stop-shop though.
> 
Well you could treat crosstool-ng as just another package that is
available to build.
So if you select crosstool-ng, buildroot automatically sets up the
correct external toolchain options.

Regards
Angus Salkeld

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Thomas Petazzoni]

Le Tue, 08 Sep 2009 23:27:41 +0200,
Peter Korsgaard <jacmet@uclibc.org> a ?crit :

> I have been thinking about it myself. *REALLY* supporting toolchains
> for all the different architectures / variants, uclibc and (e)glibc
> and following up on compiler bugs takes quite some effort (and noone
> seems willing to do it for buildroot), so maybe the best long term
> solution is simply to deprecate the toolchain building stuff and just
> recommend crosstool-ng instead?

That was my plan...

> This would make BR less selfcontained / one-stop-shop though.

until I met Buildroot users (I mean not end-users but organizations
using Buildroot as the build system they ship with their devices) who
told me that the selfcontained nature of BR was something they really
liked.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Thomas Petazzoni]

Le Wed, 09 Sep 2009 09:51:32 +1200,
"angus salkeld" <angus.salkeld@alliedtelesis.co.nz> a ?crit :

> Well you could treat crosstool-ng as just another package that is
> available to build.
> So if you select crosstool-ng, buildroot automatically sets up the
> correct external toolchain options.

That's my long term plan. Introduce a new toolchain type in Buildroot:
Crosstool-NG toolchain. Just like for uClibc today, Buildroot would
produce/tune the Crosstool-NG configuration before running the
Crosstool-NG compilation process. We could have this in parallel to the
internal building system for toolchains for a while, and then when
we're satisfied, switch to Crosstool-NG completely by removing the old
mechanism.

Sincerly,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Thomas Petazzoni]

Hello,

Thanks for your comments!

Le Wed, 09 Sep 2009 07:54:25 +1200,
"angus salkeld" <angus.salkeld@alliedtelesis.co.nz> a ?crit :

> >  2. Remove the BOARD/LOCAL feature.
> > 
> >     This mechanism is a duplication of something that could already
> > be done in a different way: by adding a new target in
> > target/device/.
> > 
> Quite a few of our engineers like this feature.
> When you are testing that a change doesn't break other targets it is
> nice to have a shell per "board" and just be able to run "make" in
> each to re-build.

Let's say your vendor name is Foo and you have two boards Bar1 and
Bar2. Then you would create the following directories:

 target/device/foo/bar1
 target/device/foo/bar2

In these directories you would have :

 * foo_bar1_defconfig and foo_bar2_defconfig, which are the default
   Buildroot configuration for the two boards

 * The Linux kernel configuration file

 * Optionnaly, a Busybox configuration file

 * Optionnaly, a target skeleton and a device table

Then, to build a board, you would just do :

 make foo_bar1_defconfig
 make

or for the other board:

 make foo_bar2_defconfig
 make

This is a mechanism that currently exists, you can try it by yourself
to verify it matches your requirements. I think the BOARD/LOCAL feature
is just a duplication of this.

Of course, there's an obvious lack of documentation on this mechanism.
But this is something I'm working on currently.

> >      - build/    where all the packages are built
> >      - images/   where the final kernel and rootfs images are stored
> >      - staging/  the staging directory (containing the development
> > files and libraries compiled for the target)
> >      - target/   which contains the target root filesystem
> >      - host/     which contains all the host programs
> >      - stamps/   which contains the stamps files
> > 
> 
> We have a a number of boards to build and this is going put the burden
> on the end user to handle moving the output directory around.
> Not very user friendly for people who use buildroot in anything but
> the simplest case.
> 
> Please still support the ability to build multiple targets within the
> same buildroot (in configuration). One option would be to make
> "O=<dir>" a configuration option (defaulting to the output directory).

That could be an option, if that makes it easier to use for your users.

Sincerly,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Peter Korsgaard]

>>>>> "angus" == angus salkeld <angus.salkeld@alliedtelesis.co.nz> writes:

Hi,

 >> This would make BR less selfcontained / one-stop-shop though.
 >> 
 angus> Well you could treat crosstool-ng as just another package that
 angus> is available to build.  So if you select crosstool-ng, buildroot
 angus> automatically sets up the correct external toolchain options.

Yes, that sounds like a nice idea. I'll take a closer look at
crosstool-ng in the near future to see how easy this would be..

-- 
Bye, Peter Korsgaard

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 Thomas> That was my plan...

 >> This would make BR less selfcontained / one-stop-shop though.

 Thomas> until I met Buildroot users (I mean not end-users but organizations
 Thomas> using Buildroot as the build system they ship with their devices) who
 Thomas> told me that the selfcontained nature of BR was something they really
 Thomas> liked.

Yes, I know - I like it myself ;) But maybe integrating crosstool-ng
support would be good enough?

-- 
Bye, Peter Korsgaard

[Buildroot] [RFC] [PATCH 0/5] Buildroot cleanup

[Will Newton]

On Tue, Sep 8, 2009 at 10:20 PM, Peter Korsgaard<jacmet@uclibc.org> wrote:
>>>>>> "Will" == Will Newton <will.newton@gmail.com> writes:
>
> Hi,
>
> ?>> ?4. Remove BR2_TOPDIR_PREFIX and BR2_TOPDIR_SUFFIX
> ?>>
> ?>> ? ?The same thing can be done with O=
>
> ?Will> I didn't know this existed either!
>
> It's documented though:
>
> http://buildroot.net/buildroot.html#building_out_of_tree

I was referring to the TOPDIR prefix and suffix options. Having delved
into the makefiles it looks to me like O= is a better option.