[Buildroot] Topics for the Buildroot Developers meeting

[Buildroot] Topics for the Buildroot Developers meeting

[Thomas Petazzoni]

Hello all,

This message is not only for the developers who will participate to the
upcoming Buildroot Developers meeting, but for all Buildroot
contributors, users and developers.

We are currently putting together a list of items to discuss/work on
during this two days meeting that will take place in Barcelona next
week-end. The list of topics we have for now is
http://elinux.org/Buildroot:DeveloperDaysELCE2012#List_of_topics_to_discuss.

Please let us know if you have other topics that you would like us to
discuss: issues with Buildroot, missing features, anything. The best is
to reply to this e-mail because the Wiki requires a login and due to
spam, they have disabled the creation of new accounts for now.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Topics for the Buildroot Developers meeting

[Danomi Manchego]

On Mon, Oct 29, 2012 at 2:43 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Please let us know if you have other topics that you would like us to
> discuss: issues with Buildroot, missing features, anything.

Maybe a couple of small items, if the attendants are interested:

- If the autobuilder emails are generated from some kind of script,
then it would be nice if the report could be modified to include
per-package failure counts.  This gives anyone who is interested in
reducing failure count an idea of what issue might reduce the failure
count the most.  For example, the following line comes close:

  $ grep "Failure reason" 2012-09-26.htm | sort | uniq -c | sort -nr
      5 Failure reason : linux-pam-1.1.4
      4 Failure reason : genext2fs-1.4.1
      3 Failure reason : lua-5.1.5
      3 Failure reason : libffi-3.0.11
      3 Failure reason : gnutls-2.12.20
      2 Failure reason : x11vnc-0.9.13
      2 Failure reason : pulseaudio-2.0
      2 Failure reason : mplayer-1.1
      2 Failure reason : icu-4.8.1.1
      2 Failure reason : directfb-1.4.16
      1 Failure reason : xenomai-2.6.0
      1 Failure reason : qt-4.8.3
      1 Failure reason : python-2.7.2

- Back in May or so, ccache.mk was changed to set the cache directory
default to an environment variable setting rather than hard-coding to
the defconfig's setting, and the Makefile modified to export that same
variable.  But when ccache is invoked by hand rather than by Makefile,
ccache uses ~/.ccache for the directory unless you set up a
BUILDROOT_CACHE_DIR environment variable that hopefully matches the
defconfig's setting.  It's nice that we can change the cache dir
without recompiling ccache (I guess), but a side effect is that it is
now more difficult to configure ccache for a larger cache (the -M
operation).  I have a project that exceeds the default 1GB default, so
I hit this issue often.  (I also like to use the -s to see status
without incurring the 3 or 4 seconds required to read all the
Makefile/*.mk files when I call "make ccache-stats" on my little Acer
Aspire netbook.)  So I was considering adding something like this, to
change ccache's last-ditch .ccache default to at least give the
possibility of matching a likely "$(HOME)/*" defconfig setting:

# Change hard-coded default to match path in the defconfig
ifneq (,$(filter "$(HOME)/%,$(BR2_CCACHE_DIR)))
  define HOST_CCACHE_CHANGE_DEFAULT_DIR
	sed -i 's,"%s/.ccache",$(BR2_CCACHE_DIR:"$(HOME)/%="\%s/%),' $(@D)/ccache.c
  endef
  HOST_CCACHE_POST_CONFIGURE_HOOKS += HOST_CCACHE_CHANGE_DEFAULT_DIR
endif

So if attendants also use ccache and have similar concerns, maybe a
minute or two can be spared to see if this is a general problem worth
looking into?

Thanks,
Danomi -

[Buildroot] Topics for the Buildroot Developers meeting

[Shawn Goff]

From the discussion page:
> Make it possible to do a 'target-clean', as a replacement of uninstall. Will this even work at all? Proposed by Arnout Vandecappelle

This would be amazing.

I had a quick chat in #ptxdist today, and it seems that's what they do
- they have separate stages, including a compile, install, and
targetinstall. The install installs the package into a
project-specific directory, and targetinstall creates an ipkg and
installs the package's relevant files to the root filesystem. When you
need to remove a package, you unselect it from the config and rebuild
the rootfs.

[Buildroot] Topics for the Buildroot Developers meeting

[Thomas Petazzoni]

On Mon, 29 Oct 2012 18:30:01 -0400, Shawn Goff wrote:
> From the discussion page:
> > Make it possible to do a 'target-clean', as a replacement of uninstall. Will this even work at all? Proposed by Arnout Vandecappelle
> 
> This would be amazing.
> 
> I had a quick chat in #ptxdist today, and it seems that's what they do
> - they have separate stages, including a compile, install, and
> targetinstall. The install installs the package into a
> project-specific directory, and targetinstall creates an ipkg and
> installs the package's relevant files to the root filesystem. When you
> need to remove a package, you unselect it from the config and rebuild
> the rootfs.

Except that it doesn't work properly, even in PTXdist (as far as I
know, of course). We had a discussion about this last year at ELCE with
PTXdist developers. Basically, do the following:

 * Build program foo with OpenSSL support (the OpenSSL support in foo
   is optional, foo can work without OpenSSL). Both foo and openssl are
   installed in the target.

 * Enjoy foo with OpenSSL on your target, you're happy.

 * Now, remove OpenSSL from your target.

Beng, "foo" no longer works, because a library it is linked against no
longer exists, and "foo" has not been rebuilt without OpenSSL support.
As far as I know, PTXdist doesn't keep track of reverse dependencies
when removing a package, and that can lead to invalid root filesystems.

That's the precise reason why we decided /not/ to support packages in
Buildroot: solving this problem completely is very complicated, and we
thought it is better to not solve the problem than to solve it in half.

We detailed all this in our ELCE 2011 meeting report, but I guess we
should make it part of our FAQ because it is actually a FAQ :-)

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Topics for the Buildroot Developers meeting

[Shawn Goff]

On Mon, Oct 29, 2012 at 6:38 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
>
> On Mon, 29 Oct 2012 18:30:01 -0400, Shawn Goff wrote:
>> From the discussion page:
>> > Make it possible to do a 'target-clean', as a replacement of uninstall. Will this even work at all? Proposed by Arnout Vandecappelle
>>
>> This would be amazing.
>>
>> I had a quick chat in #ptxdist today, and it seems that's what they do
>> - they have separate stages, including a compile, install, and
>> targetinstall. The install installs the package into a
>> project-specific directory, and targetinstall creates an ipkg and
>> installs the package's relevant files to the root filesystem. When you
>> need to remove a package, you unselect it from the config and rebuild
>> the rootfs.
>
> Except that it doesn't work properly, even in PTXdist (as far as I
> know, of course). We had a discussion about this last year at ELCE with
> PTXdist developers. Basically, do the following:
>
>  * Build program foo with OpenSSL support (the OpenSSL support in foo
>    is optional, foo can work without OpenSSL). Both foo and openssl are
>    installed in the target.
>
>  * Enjoy foo with OpenSSL on your target, you're happy.
>
>  * Now, remove OpenSSL from your target.
>
> Beng, "foo" no longer works, because a library it is linked against no
> longer exists, and "foo" has not been rebuilt without OpenSSL support.
> As far as I know, PTXdist doesn't keep track of reverse dependencies
> when removing a package, and that can lead to invalid root filesystems.
>
> That's the precise reason why we decided /not/ to support packages in
> Buildroot: solving this problem completely is very complicated, and we
> thought it is better to not solve the problem than to solve it in half.

Yes, but I (and probably 90% or so?) of the people who use Buildroot
are aware enough to know that yanking out a dependency can mess with
things. And when the problem occurs, 99% of the people who use
buildroot can solve it by themselves when the problem happens. There
is even a very well-known distribution that people actually use as
their main system that doesn't do any dependency checking.

Maybe it could be an undocumented target? Maybe it could be documented
with big warnings around it?

[Buildroot] Topics for the Buildroot Developers meeting

[Robert Schwebel]

On Mon, Oct 29, 2012 at 11:38:15PM +0100, Thomas Petazzoni wrote:
> Except that it doesn't work properly, even in PTXdist (as far as I
> know, of course). We had a discussion about this last year at ELCE with
> PTXdist developers. Basically, do the following:
>
>  * Build program foo with OpenSSL support (the OpenSSL support in foo
>    is optional, foo can work without OpenSSL). Both foo and openssl are
>    installed in the target.
>
>  * Enjoy foo with OpenSSL on your target, you're happy.
>
>  * Now, remove OpenSSL from your target.
>
> Beng, "foo" no longer works, because a library it is linked against no
> longer exists, and "foo" has not been rebuilt without OpenSSL support.
> As far as I know, PTXdist doesn't keep track of reverse dependencies
> when removing a package, and that can lead to invalid root filesystems.

You are basically right, but ptxdist's intention is to make reproducable
systems, which means that a system stays consistent to itself when you

a) check out the BSP from your repository
b) build it from scratch

That's the only guarantee it gives you. There is no promise that
anything stays consistent if you configure around while a half-built
system is already there.

Because we have dependencies derived from the "select" statements in
Kconfig, it most times works pretty well if you switch things on, which
is what people most often do while developing.

If you switch things off, packages might become inconsistent, but again,
there is no promise that this works.

Any idea for a better mechanism is welcome :-)

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |