[Buildroot] Recommended way of working on project

[Buildroot] Recommended way of working on project

[David Kosir]

Hi guys,

I need advice from someone who already did real-life projects with buildroot.

I've started a buildroot project and I'm wondering what is the best
way of managing it.
I will mostly use open-source packages that are already part of the
buildroot, but I will need to update versions and change specific
build options on many of them. Also, I will add few custom
scripts/programs.

I was thinking what is the best way of handling it. I've read
recommendations here [1]
I see two recommended ways and third non-recommended but often approach:

1) Use company-specific configuration and packages, don't interfere
with buildroot files.
2) Use BR2_EXTERNAL and don't touch buildroot (probably use it as git
submodule).
3) Fork buildroot and change whatever you like. Fetch new changes
now-and-then, if possible to merge easily.

I know that 1) and 2) are recommended but I've already stumbled upon
problem keeps growing and repeating:

- I need to use XY package but I need newer version
- I create new XY-custom package in my <company>/<board> or BR2_EXTERNAL path
- XY package is dependency for few other packages so I need to create
them as *-custom package
- I end up with 10 custom packages in two days and I expect to have 10
more next days.

As my final product will basically be using all my custom packages +
few basic buildroot packages, is there and good reason not to go with
approach no 3) ?
I see it often online, when people just fork buildroot and continue
working on some stable tag, rarely even bothering on merging from
mainline as their image works and there is no need to touch it.

While 1) and 2) look like "right" way of doing it (as they give easy
buildroot merging) it feels like running parallel "mini-buildroot"
copying mainline packages and doing minor changes to them.

When time comes to fetch new changes, in 1) and 2) I will need to
looks for changes manually as I've copied old packages and in 3) I
will have to do manual merge examining conflicts.

What are your real life project experiences, what is the best way?

Thanks
David

[1] http://buildroot.uclibc.org/downloads/manual/manual.html#customize

[Buildroot] Recommended way of working on project

[Gabe Evans]

Hi David,

I've used several strategies for different projects we've deployed in
production and BR2_EXTERNAL is what I've settled on. It's by far the best
option for the majority of projects. I would only recommend forking
Buildroot when making changes you plan to merge upstream. Otherwise,
you're likely to make upgrading to future releases difficult.

Is there a specific package you're trying to tweak that's resulted in
you having to make changes in multiple packages depending on it?

Some of the ways I've added customizations to a package without necessarily
copying the entirety into my own tree include:

1. Making small tweaks to code by using patches in BR2_GLOBAL_PATCH_DIR.

2. Using a different source by creating a custom package that only adjusts
   the <PACKAGE>_SOURCE/SITE of the original package.

3. Adding an additional build step to a package by doing the same as #2 to
   add a pre/post hook to the original package.

This has made maintenance trivial and in cases where there's a conflict,
I'm able to focus my attention on a couple lines instead of on the
entirety of the package. Hopefully others can chime in on any other
pitfalls there could be with this strategy.

Hope this helps!

-Gabe


On Mon, Oct 26, 2015 at 2:56 PM, David Kosir <david.kosir@bylapis.com> wrote:
> Hi guys,
>
> I need advice from someone who already did real-life projects with buildroot.
>
> I've started a buildroot project and I'm wondering what is the best
> way of managing it.
> I will mostly use open-source packages that are already part of the
> buildroot, but I will need to update versions and change specific
> build options on many of them. Also, I will add few custom
> scripts/programs.
>
> I was thinking what is the best way of handling it. I've read
> recommendations here [1]
> I see two recommended ways and third non-recommended but often approach:
>
> 1) Use company-specific configuration and packages, don't interfere
> with buildroot files.
> 2) Use BR2_EXTERNAL and don't touch buildroot (probably use it as git
> submodule).
> 3) Fork buildroot and change whatever you like. Fetch new changes
> now-and-then, if possible to merge easily.
>
> I know that 1) and 2) are recommended but I've already stumbled upon
> problem keeps growing and repeating:
>
> - I need to use XY package but I need newer version
> - I create new XY-custom package in my <company>/<board> or BR2_EXTERNAL path
> - XY package is dependency for few other packages so I need to create
> them as *-custom package
> - I end up with 10 custom packages in two days and I expect to have 10
> more next days.
>
> As my final product will basically be using all my custom packages +
> few basic buildroot packages, is there and good reason not to go with
> approach no 3) ?
> I see it often online, when people just fork buildroot and continue
> working on some stable tag, rarely even bothering on merging from
> mainline as their image works and there is no need to touch it.
>
> While 1) and 2) look like "right" way of doing it (as they give easy
> buildroot merging) it feels like running parallel "mini-buildroot"
> copying mainline packages and doing minor changes to them.
>
> When time comes to fetch new changes, in 1) and 2) I will need to
> looks for changes manually as I've copied old packages and in 3) I
> will have to do manual merge examining conflicts.
>
> What are your real life project experiences, what is the best way?
>
> Thanks
> David
>
> [1] http://buildroot.uclibc.org/downloads/manual/manual.html#customize
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Gabe Evans | Co-Founder & CTO
hashrabbit.co ? angel.co/hashrabbit ? github.com/gevans

[Buildroot] Recommended way of working on project

[Arnout Vandecappelle]

On 26-10-15 22:56, David Kosir wrote:
> Hi guys,
> 
> I need advice from someone who already did real-life projects with buildroot.
> 
> I've started a buildroot project and I'm wondering what is the best
> way of managing it.
> I will mostly use open-source packages that are already part of the
> buildroot, but I will need to update versions and change specific
> build options on many of them. Also, I will add few custom
> scripts/programs.
> 
> I was thinking what is the best way of handling it. I've read
> recommendations here [1]
> I see two recommended ways and third non-recommended but often approach:
> 
> 1) Use company-specific configuration and packages, don't interfere
> with buildroot files.
> 2) Use BR2_EXTERNAL and don't touch buildroot (probably use it as git
> submodule).
> 3) Fork buildroot and change whatever you like. Fetch new changes
> now-and-then, if possible to merge easily.

 Patching buildroot itself is not recommended, but it's not discouraged either.
Or did you find somewhere advising against it? In fact, some of the core
buildroot developers (Peter, me, and I think Thomas as well) regularly do that
in our buildroot projects. The approach 1) and 2) are really only about adding
packages.

 What we do discourage is public forks that try to add support for some specific
board, because these things should just be upstreamed.


> I know that 1) and 2) are recommended but I've already stumbled upon
> problem keeps growing and repeating:
> 
> - I need to use XY package but I need newer version
> - I create new XY-custom package in my <company>/<board> or BR2_EXTERNAL path
> - XY package is dependency for few other packages so I need to create
> them as *-custom package
> - I end up with 10 custom packages in two days and I expect to have 10
> more next days.

 Yep, gets complicated :-)

> 
> As my final product will basically be using all my custom packages +
> few basic buildroot packages, is there and good reason not to go with
> approach no 3) ?
> I see it often online, when people just fork buildroot and continue
> working on some stable tag, rarely even bothering on merging from
> mainline as their image works and there is no need to touch it.
> 
> While 1) and 2) look like "right" way of doing it (as they give easy
> buildroot merging) it feels like running parallel "mini-buildroot"
> copying mainline packages and doing minor changes to them.
> 
> When time comes to fetch new changes, in 1) and 2) I will need to
> looks for changes manually as I've copied old packages and in 3) I
> will have to do manual merge examining conflicts.
> 
> What are your real life project experiences, what is the best way?


 In fact, I typically use a combination of 2) and 3). New "private" packages go
into BR2_EXTERNAL, new to-be-upstreamed packages and local hacks are applied
directly on the buildroot tree. Merging upstream changes is usually not much of
a problem.

 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Recommended way of working on project

[Arnout Vandecappelle]

On 27-10-15 21:11, Gabe Evans wrote:
> Hi David,
> 
> I've used several strategies for different projects we've deployed in
> production and BR2_EXTERNAL is what I've settled on. It's by far the best
> option for the majority of projects. I would only recommend forking
> Buildroot when making changes you plan to merge upstream. Otherwise,
> you're likely to make upgrading to future releases difficult.
> 
> Is there a specific package you're trying to tweak that's resulted in
> you having to make changes in multiple packages depending on it?
> 
> Some of the ways I've added customizations to a package without necessarily
> copying the entirety into my own tree include:
> 
> 1. Making small tweaks to code by using patches in BR2_GLOBAL_PATCH_DIR.

 That's indeed another good way to make changes, but doesn't help if you want to
change some configure or install option.

> 
> 2. Using a different source by creating a custom package that only adjusts
>    the <PACKAGE>_SOURCE/SITE of the original package.

 Note that this is not possible for _VERSION, _INSTALL_STAGING/TARGET,
_SITE_METHOD and probably a few others.

> 3. Adding an additional build step to a package by doing the same as #2 to
>    add a pre/post hook to the original package.
> 
> This has made maintenance trivial and in cases where there's a conflict,
> I'm able to focus my attention on a couple lines instead of on the
> entirety of the package.

 But doesn't that make things more complicated? I mean, you don't have an
overview of what's really happening, and when you look at the package.mk file
you don't see those overrides so you don't understand why it's behaving differently.

 Also, when you merge with upstream and something relevant has changed, there
will _not_ be a merge conflict but instead things go wrong in unexpected ways.

 So I'm not in favour of this approach. But if it works for you, by all means
continue doing so!

 Regards,
 Arnout

> Hopefully others can chime in on any other
> pitfalls there could be with this strategy.
> 

[snip]
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Recommended way of working on project

[Carlos Santos]

> From: "David Kosir" <david.kosir@bylapis.com>
> To: buildroot at buildroot.org
> Sent: Monday, October 26, 2015 7:56:34 PM
> Subject: [Buildroot] Recommended way of working on project

[...]

> I was thinking what is the best way of handling it. I've read
> recommendations here [1]

Here at DATACOM we have a rather large project with 144 local modules. Most of them are developed locally but we also have third-party code, ether proprietary or open source. We use Gerrit for source code management and review.

Our recipes download code from Gerrit. There is also a local cache of external code, pointed-out by BR2_PRIMARY_SITE. This speeds up the builds, since we don't need to access external sites (e.g. GitHub, kernel.org) to get the code.

We use custom toolchains,  according to the target platform architecture. They are installed at a fixed path (BR2_TOOLCHAIN_EXTERNAL_PATH=/opt/<foo>) to speed up build start-up.

Each module has a unit test (tier 1 test). Continuous integration is orchestrated by Jenkis. Before integration each module is submitted to static code analysis (tier 2 tests) and a quite large amount of automated functional tests (tier 3). This may sound fancy but in fact we are just playing by the book using widely known open source and commercial tools like GMock/GTest, CodeSonar and Robot Framework. Of course it requires a lot of work and duct tape.

> I see two recommended ways and third non-recommended but often approach:
> 
> 1) Use company-specific configuration and packages, don't interfere
> with buildroot files.
> 2) Use BR2_EXTERNAL and don't touch buildroot (probably use it as git
> submodule).

These options are equivalent but (2) allows us to keep our recipes in a separate repository. This is code that we can't share either because it is proprietary or because i would be useless outside so we must maintain it ourselves.

> 3) Fork buildroot and change whatever you like. Fetch new changes
> now-and-then, if possible to merge easily.

We make few changes on Buildroot but always attempt to submit them upstream. We keep a local replica on which we apply these changes on a company-local branch while their integration upstream is under work. As soon as a new Buildroot release is made available we rebase our code, removing all local additions that have been integrated.

You cam see the result of this work by looking for "@datacom.ind.br" in the master branch log:

    $ git log --oneline --author=@datacom.ind.br master

> I know that 1) and 2) are recommended but I've already stumbled upon
> problem keeps growing and repeating:
> 
> - I need to use XY package but I need newer version

We upgrade the package in our local replica and submit the update upstream.

> - I create new XY-custom package in my <company>/<board> or BR2_EXTERNAL path

We create it under $(BR2_EXTERNAL)/package and add FOO_BAR=y to

   $(BR2_EXTERNAL)/configs/$(BOAR_NAME)_defconfig

> - XY package is dependency for few other packages so I need to create
> them as *-custom package

For development we to this:

  - add a recipe under $(BR2_EXTERNAL)/package

  - add a line to $(BR2_EXTERNAL)/Config.in containing

    source "$BR2_EXTERNAL/package/foo-bar/Config.in"

  - tweak $(BR2_EXTERNAL)/configs/$(BOAR_NAME)_defconfig, adding

    FOO_BAR=y

  - add "FOO_BAR_OVERRIDE_SRCDIR = <path>" to local.mk (see documentation) for local development

After the new module is ready for integration we send it to its own Gerrit project and update the BR2_EXTERNAL project accordingly

> - I end up with 10 custom packages in two days and I expect to have 10
> more next days.

Repeat previous step 10 times. If you have different teams working on each module you will need to coordinate them but it is feasible. We did if for 154 modules (and counting).

> As my final product will basically be using all my custom packages +
> few basic buildroot packages, is there and good reason not to go with
> approach no 3) ?
> 
> I see it often online, when people just fork buildroot and continue
> working on some stable tag, rarely even bothering on merging from
> mainline as their image works and there is no need to touch it.

I'd strongly recommend you to avoid forking Buildroot. This option looks easier at first sight but becomes a nightmare in the future, when you attempt to rebase it. BR2_EXTERNAL is your friend!

> While 1) and 2) look like "right" way of doing it (as they give easy
> buildroot merging) it feels like running parallel "mini-buildroot"
> copying mainline packages and doing minor changes to them.

Do not copy/change mainline packages unless you intend to send the customizations upstream. Each new package you create is a baby that you will need to care about indefinitely. A simpler alternative is to keep custom patches under

   $(BR2_EXTERNAL)/board/all/patches/<pkg-name>[/<pkg-version>]
   $(BR2_EXTERNAL)/board/$(BOARD_NAME)/patches/<pkg-name>[/<pkg-version>]

Again, follow the documentation.

> When time comes to fetch new changes, in 1) and 2) I will need to
> looks for changes manually as I've copied old packages and in 3) I
> will have to do manual merge examining conflicts.

Keep your improvements in a local replica and submit them upstream. Rebase periodically (e.g. at each stable release).

> What are your real life project experiences, what is the best way?

If it were easy my 83 years old mommy would be doing it. Anyway, it's not rocket science; just requires strong discipline and careful planning. Endure the pain and enjoy the gain!

> Thanks
> David
> 
> [1] http://buildroot.uclibc.org/downloads/manual/manual.html#customize


Carlos Santos (Casantos)
DATACOM, P&D