[Buildroot] uboot-tools and uboot being separate

[Buildroot] uboot-tools and uboot being separate

[Dimitrios Siganos]

Hi guys,

I have a query regarding uboot and uboot-tools. Currently they are
separate packages.

However, if I am building both the uboot bootloader and the uboot tools
would it not be reasonable to expect to use the same sources for
compiling both? At the moment, I am in a situation where I am building
uboot with one set of files and uboot-tools (e.g. fw_printenv) with another.

Is the recommended solution to point both uboot and uboot-tools to the
same package version and apply the same patches to both? And have two
parallel builds (uboot, uboot-tools) that should be identical?

I have also being thinking along these lines:

1) Edit the uboot-tools makefile to not use its own build directory but
use uboot's build instead (silly idea idea I admit, but you never know...).

2) Edit the uboot package to optionally compile and install the
uboot-tools as well (this seems to me to be the most logical way).

Can I have your feedback?

Regards,
Dimitris

[Buildroot] uboot-tools and uboot being separate

[Thomas Petazzoni]

Dear Dimitrios Siganos,

On Wed, 13 Feb 2013 17:22:54 +0000, Dimitrios Siganos wrote:

> I have a query regarding uboot and uboot-tools. Currently they are
> separate packages.
> 
> However, if I am building both the uboot bootloader and the uboot tools
> would it not be reasonable to expect to use the same sources for
> compiling both? At the moment, I am in a situation where I am building
> uboot with one set of files and uboot-tools (e.g. fw_printenv) with another.
> 
> Is the recommended solution to point both uboot and uboot-tools to the
> same package version and apply the same patches to both? And have two
> parallel builds (uboot, uboot-tools) that should be identical?

No. You don't have to have identical uboot-tools and uboot. Basically
uboot-tools provide one tool to create U-Boot images (mkimage) and
tools to manipulate the U-Boot environment from Linux (fw_printenv,
fw_setenv). Those tools are backward compatible, and so you can
perfectly use the tools from U-Boot 2010.x with a running U-Boot 2012.x
or 2013.x.

There is really no need to have the same source code base for both
uboot and uboot-tools.

> I have also being thinking along these lines:
> 
> 1) Edit the uboot-tools makefile to not use its own build directory but
> use uboot's build instead (silly idea idea I admit, but you never know...).
> 
> 2) Edit the uboot package to optionally compile and install the
> uboot-tools as well (this seems to me to be the most logical way).
> 
> Can I have your feedback?

Basically, no, there's a good reason why we wanted two separate
packages: we wanted to be able to build the U-Boot tools sometimes for
the host (mkimage), sometimes for the target (fw_printenv, fw_setenv).
The host-uboot-tools package is also used as a dependency of the Linux
kernel package, and it is much simpler to depend on host-uboot-tools
that to depend on the uboot package itself.

So really, the current way things are done for uboot vs. uboot-tools
has proven to work really well, and for now I don't see a compelling
reason to change that.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] uboot-tools and uboot being separate

[Arnout Vandecappelle]

On 13/02/13 21:20, Thomas Petazzoni wrote:
> Dear Dimitrios Siganos,
>
> On Wed, 13 Feb 2013 17:22:54 +0000, Dimitrios Siganos wrote:
>
>> I have a query regarding uboot and uboot-tools. Currently they are
>> separate packages.
>>
>> However, if I am building both the uboot bootloader and the uboot tools
>> would it not be reasonable to expect to use the same sources for
>> compiling both? At the moment, I am in a situation where I am building
>> uboot with one set of files and uboot-tools (e.g. fw_printenv) with another.
>>
>> Is the recommended solution to point both uboot and uboot-tools to the
>> same package version and apply the same patches to both? And have two
>> parallel builds (uboot, uboot-tools) that should be identical?

  I've had the same issue once: a patch to add some functionality to 
mkimage that I needed both on the host and on the target.

>
> No. You don't have to have identical uboot-tools and uboot. Basically
> uboot-tools provide one tool to create U-Boot images (mkimage) and
> tools to manipulate the U-Boot environment from Linux (fw_printenv,
> fw_setenv). Those tools are backward compatible, and so you can
> perfectly use the tools from U-Boot 2010.x with a running U-Boot 2012.x
> or 2013.x.

  Unless some feature has been added or removed.

>
> There is really no need to have the same source code base for both
> uboot and uboot-tools.
>
>> I have also being thinking along these lines:
>>
>> 1) Edit the uboot-tools makefile to not use its own build directory but
>> use uboot's build instead (silly idea idea I admit, but you never know...).

  This will be the way to go - and much easier as soon as the out-of-tree 
build is there. But it still requires more infrastructural change, since 
currently there is no way to specify that a package doesn't have a source 
itself, but the source comes from another package.

  There are actually a few other packages out there that have or could 
have the same "shared source, separate build" issue: kernel-headers, 
udev, perf.


  Regards,
  Arnout

>>
>> 2) Edit the uboot package to optionally compile and install the
>> uboot-tools as well (this seems to me to be the most logical way).
>>
>> Can I have your feedback?
>
> Basically, no, there's a good reason why we wanted two separate
> packages: we wanted to be able to build the U-Boot tools sometimes for
> the host (mkimage), sometimes for the target (fw_printenv, fw_setenv).
> The host-uboot-tools package is also used as a dependency of the Linux
> kernel package, and it is much simpler to depend on host-uboot-tools
> that to depend on the uboot package itself.
>
> So really, the current way things are done for uboot vs. uboot-tools
> has proven to work really well, and for now I don't see a compelling
> reason to change that.
>
> Best regards,
>
> Thomas
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] uboot-tools and uboot being separate

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Wed, 13 Feb 2013 23:57:03 +0100, Arnout Vandecappelle wrote:

>   I've had the same issue once: a patch to add some functionality to 
> mkimage that I needed both on the host and on the target.

Then easy enough; put your patch in package/uboot-tools/. It handles
the build and installation of both the host *and* target variant of the
U-Boot tools.

> > No. You don't have to have identical uboot-tools and uboot.
> > Basically uboot-tools provide one tool to create U-Boot images
> > (mkimage) and tools to manipulate the U-Boot environment from Linux
> > (fw_printenv, fw_setenv). Those tools are backward compatible, and
> > so you can perfectly use the tools from U-Boot 2010.x with a
> > running U-Boot 2012.x or 2013.x.
> 
>   Unless some feature has been added or removed.

In which case you bump the version of the uboot-tools package, just
like any other package.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] uboot-tools and uboot being separate

[Arnout Vandecappelle]

On 14/02/13 00:14, Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
>
> On Wed, 13 Feb 2013 23:57:03 +0100, Arnout Vandecappelle wrote:
>
>>    I've had the same issue once: a patch to add some functionality to
>> mkimage that I needed both on the host and on the target.
>
> Then easy enough; put your patch in package/uboot-tools/. It handles
> the build and installation of both the host *and* target variant of the
> U-Boot tools.

  In my particular use case (IIRC), the patched mkimage was used to 
process u-boot.bin into a ROM-bootable image. So the same patch had to be 
applied to uboot-tools and uboot. But it was not in fact the same patch, 
because the uboot version was different. What we did in the end is just 
change the uboot-tools site to the same git repository.


>>> No. You don't have to have identical uboot-tools and uboot.
>>> Basically uboot-tools provide one tool to create U-Boot images
>>> (mkimage) and tools to manipulate the U-Boot environment from Linux
>>> (fw_printenv, fw_setenv). Those tools are backward compatible, and
>>> so you can perfectly use the tools from U-Boot 2010.x with a
>>> running U-Boot 2012.x or 2013.x.
>>
>>    Unless some feature has been added or removed.
>
> In which case you bump the version of the uboot-tools package, just
> like any other package.

  Yes, but you have to do it twice.

  It's not that it's impossible to solve, it's just inconvenient and 
counter-intuitive. It's also inconvenient to have two U-Boot versions in 
your legal-info output. Just like it's inconvenient to have two linux 
tarballs.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] uboot-tools and uboot being separate

[Dimitrios Siganos]

On 13/02/13 23:21, Arnout Vandecappelle wrote:
> On 14/02/13 00:14, Thomas Petazzoni wrote:
>> Dear Arnout Vandecappelle,
>>
>> On Wed, 13 Feb 2013 23:57:03 +0100, Arnout Vandecappelle wrote:
>>
>>>    I've had the same issue once: a patch to add some functionality to
>>> mkimage that I needed both on the host and on the target.
>>
>> Then easy enough; put your patch in package/uboot-tools/. It handles
>> the build and installation of both the host *and* target variant of the
>> U-Boot tools.
> 
>  In my particular use case (IIRC), the patched mkimage was used to
> process u-boot.bin into a ROM-bootable image. So the same patch had to
> be applied to uboot-tools and uboot. But it was not in fact the same
> patch, because the uboot version was different. What we did in the end
> is just change the uboot-tools site to the same git repository.

Hi Thomas, Arnout,

So if I understand this well, the consensus is that the preferred way to
solve this is to set both packages to point to the same site, method and
version and have two identical copies of the same uboot source code, one
in uboot package and one in uboot-tools package.

That certainly sounds the simplest way although I agree with Arnout that
it is also the strangest and less intuitive approach. Some people will
still raise an eyebrow at such an arrangement.

Regards,
Dimitris

[Buildroot] uboot-tools and uboot being separate

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Thu, 14 Feb 2013 00:21:12 +0100, Arnout Vandecappelle wrote:

>   It's not that it's impossible to solve, it's just inconvenient and 
> counter-intuitive. It's also inconvenient to have two U-Boot versions
> in your legal-info output. Just like it's inconvenient to have two
> linux tarballs.

Indeed, but what solution do you propose?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] uboot-tools and uboot being separate

[Dimitrios Siganos]

On 14/02/13 07:57, Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
> 
> On Thu, 14 Feb 2013 00:21:12 +0100, Arnout Vandecappelle wrote:
> 
>>   It's not that it's impossible to solve, it's just inconvenient and 
>> counter-intuitive. It's also inconvenient to have two U-Boot versions
>> in your legal-info output. Just like it's inconvenient to have two
>> linux tarballs.
> 
> Indeed, but what solution do you propose?
> 
> Thomas

The ideal scenario that I am picturing is this.

* Add an option to uboot (let's call it UBOOT_BUILD_ENV_TOOLS), to
additionally do the work that is currently done by uboot-tools.

* Add an option to uboot-tools to make it a "virtual" package (let's
call it UBOOT_TOOLS_VIRTUAL) that tells uboot-tools to do nothing
because uboot will do all the work. The virtual package is needed so
that the dependency chains can lead to uboot.

* Turning on virtual mode in uboot-tools also turns on uboot
UBOOT_BUILD_ENV_TOOLS.

* The uboot-tools virtual package would add uboot as a dependency if
UBOOT_TOOLS_VIRTUAL is on.

I do not have much hands-on experience on this so I could talking
rubbish. Please let me know if I am talking rubbish.

In the mean time, the practical way to ensure building from both sources
is to set:

UBOOT_VERSION equal to UBOOT_TOOLS_VERSION
UBOOT_SITE equal to UBOOT_TOOLS_SITE
UBOOT_METHOD equal to UBOOT_TOOLS_METHOD

However, because uboot has many more options for setting SITE and
METHOD, it is currently not always possible to set them to pull the same
sources without patching buildroot. I currently get uboot from a git repo.

If I created a patch to give uboot-tools the same pull options as the
uboot package, would you accept it (if it passes review of course)?

Regards,
Dimitris

[Buildroot] uboot-tools and uboot being separate

[Dimitrios Siganos]

On 13/02/13 20:20, Thomas Petazzoni wrote:
> Dear Dimitrios Siganos,
> 
> On Wed, 13 Feb 2013 17:22:54 +0000, Dimitrios Siganos wrote:
> 
>> I have a query regarding uboot and uboot-tools. Currently they are
>> separate packages.
>>
>> However, if I am building both the uboot bootloader and the uboot tools
>> would it not be reasonable to expect to use the same sources for
>> compiling both? At the moment, I am in a situation where I am building
>> uboot with one set of files and uboot-tools (e.g. fw_printenv) with another.
>>
>> Is the recommended solution to point both uboot and uboot-tools to the
>> same package version and apply the same patches to both? And have two
>> parallel builds (uboot, uboot-tools) that should be identical?
> 
> No. You don't have to have identical uboot-tools and uboot. Basically
> uboot-tools provide one tool to create U-Boot images (mkimage) and
> tools to manipulate the U-Boot environment from Linux (fw_printenv,
> fw_setenv). Those tools are backward compatible, and so you can
> perfectly use the tools from U-Boot 2010.x with a running U-Boot 2012.x
> or 2013.x.
> 
> There is really no need to have the same source code base for both
> uboot and uboot-tools.
> 
>> I have also being thinking along these lines:
>>
>> 1) Edit the uboot-tools makefile to not use its own build directory but
>> use uboot's build instead (silly idea idea I admit, but you never know...).
>>
>> 2) Edit the uboot package to optionally compile and install the
>> uboot-tools as well (this seems to me to be the most logical way).
>>
>> Can I have your feedback?
> 
> Basically, no, there's a good reason why we wanted two separate
> packages: we wanted to be able to build the U-Boot tools sometimes for
> the host (mkimage), sometimes for the target (fw_printenv, fw_setenv).
> The host-uboot-tools package is also used as a dependency of the Linux
> kernel package, and it is much simpler to depend on host-uboot-tools
> that to depend on the uboot package itself.
> 
> So really, the current way things are done for uboot vs. uboot-tools
> has proven to work really well, and for now I don't see a compelling
> reason to change that.

Hi Thomas,

In this case, the issue is as much political as it is technical. I have
recently moved the build system of a company to buildroot but an
engineer, who had designed the old system, is convinced that the uboot
in the new system is less reliable and he attributes that to the
different sources used to build uboot and uboot-tools. I have not
noticed an increase in unreliability but really it is impossible for us
to measure such things since we have no automated testing. The reality
is that we just don't know. However, the engineer is adamant that there
is a problem and that building from the same sources is the solution or
at least the beginning of the solution and it is hard to argue against it.

However, I appreciate that this is more of a political problem rather
than a technical one and that it is a tricky situation all around.

Regards,
Dimitris