[Buildroot] Hiding non-buildable packages

[Buildroot] Hiding non-buildable packages

[Ulf Samuelsson]

A lot of the packages does not build.
Some only build for some targets.
I think it would be nice to have the possibility
to set a config item, which resulted in that only
packages which seems to build are really visible.

A lot of people will waste time to try to build
packages which simply does not work...
openssh (or was it openssl) is but one example.

Implementation could be:



Dependencies.in:	
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
config	BR2_HIDE_BROKEN_ARM
	bool "Hide packages, known not to build on ARM"
	default n
	depends on BR2_arm 
	select	BR2_HIDE_OPENSSL
	select  BR2_HIDE_OPENSSH
	[snip]
	select  BR2_HIDE_X11R7
	help
	  Setting this flag will hide packages which 
	  for various reasons will not build on ARM CPUs.

config	BR2_HIDE_BROKEN_X86
	bool "Hide packages, known not to build on ARM"
	default n
	depends on BR2_i386 
	select  BR2_HIDE_OPENSSH
	[snip]
	help
	  Setting this flag will hide packages which 
	  for various reasons will not build on an x86 CPU

...

config	BR2_HIDE_OPENSSH
	bool
	default n

config	BR2_HIDE_OPENSSL
	bool
	default n

[snip]

config	BR2_HIDE_X11R7
	bool
	default n

--------------------------------------------------------



package/openssh/Config.in:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
config	BR2_PACKAGE_OPENSSH
	depends on !BR2_HIDE_OPENSSH
	...
--------------------------------------------------------


Didn't test this, yet, but looks useful.


-- 
Best Regards,
Ulf Samuelsson

[Buildroot] Hiding non-buildable packages

[Bernhard Fischer]

On Tue, Aug 21, 2007 at 10:28:57PM +0200, Ulf Samuelsson wrote:
>A lot of the packages does not build.

Fixing them is the right thing to do.

>Some only build for some targets.

Certain packages (think specific HW support) should depend on a specific
arch. Which are these?

>I think it would be nice to have the possibility
>to set a config item, which resulted in that only
>packages which seems to build are really visible.

I object to this if you just want to avoid helping to provide patches to
upstream that do fix the respective packages that are ment to work fine.

[Buildroot] Hiding non-buildable packages

[Ulf Samuelsson]

ons 2007-08-22 klockan 21:13 +0200 skrev Bernhard Fischer:
> On Tue, Aug 21, 2007 at 10:28:57PM +0200, Ulf Samuelsson wrote:
> >A lot of the packages does not build.
> 
> Fixing them is the right thing to do.

Yes, the goal is to have all packages working for all architectures.
We are not there by far.

> 
> >Some only build for some targets.
> 
> Certain packages (think specific HW support) should depend on a specific
> arch. Which are these?

I do not have this list.

> >I think it would be nice to have the possibility
> >to set a config item, which resulted in that only
> >packages which seems to build are really visible.
> 
> I object to this if you just want to avoid helping to provide patches to
> upstream that do fix the respective packages that are ment to work fine.

No.
Do you think I have been too passive submitting patches during the last
2 months :-)

I see this as a help to quickly determine what I can do and cannot do 
at a certain point of time. The goal is to remove the restrictions.

I am mostly interested in ARM and AVR32.
There are a lot of packages, which needs an active porting effort to 
build on the AVR32, since those packages explicitly state which 
architectures are supported, and the AVR32 might not be listed.
It would be very good if people would know up front what can be done at
the moment.

On the ARM, the problem is different. 
Packages, not involving direct access to architecure specific H/W
(mostly PC stuff) should build, but for various reasons they do not.
If people starting using buildroot had this help, then they could
spend their time first building a working filesystem, and then select
a specific, non-working package, to debug.
Today endless hours can be spent on trying to build stuff which ends in
a failure.
If it is known not to build, it is good to communicate this to others.


With this fix, it will be easy to grep package/*/Config.in 
for a list of non-working packages, and then prioritize
what to work on.



-- 
Best Regards,
Ulf Samuelsson

[Buildroot] Hiding non-buildable packages

[Jonathan Dumaresq]

Hi,

I juste want to tell you that i will need to use both arch. ARM9 (RM9200) 
and AVR32 (NGW100). So i probably try both setup. For the AVR32 should be 
easier for me to try it. For the RM9200 it's a custom board based on the 
RM9200-EK or DK i don't remember. SO this one should be not soon before i 
can get my board up and running.

Jonathan


----- Original Message ----- 
From: "Ulf Samuelsson" <ulf@atmel.com>
To: "Bernhard Fischer" <rep.dot.nop@gmail.com>
Cc: "Buildroot" <buildroot@uclibc.org>
Sent: Wednesday, August 22, 2007 4:24 PM
Subject: Re: [Buildroot] Hiding non-buildable packages


> ons 2007-08-22 klockan 21:13 +0200 skrev Bernhard Fischer:
>> On Tue, Aug 21, 2007 at 10:28:57PM +0200, Ulf Samuelsson wrote:
>> >A lot of the packages does not build.
>> 
>> Fixing them is the right thing to do.
> 
> Yes, the goal is to have all packages working for all architectures.
> We are not there by far.
> 
>> 
>> >Some only build for some targets.
>> 
>> Certain packages (think specific HW support) should depend on a specific
>> arch. Which are these?
> 
> I do not have this list.
> 
>> >I think it would be nice to have the possibility
>> >to set a config item, which resulted in that only
>> >packages which seems to build are really visible.
>> 
>> I object to this if you just want to avoid helping to provide patches to
>> upstream that do fix the respective packages that are ment to work fine.
> 
> No.
> Do you think I have been too passive submitting patches during the last
> 2 months :-)
> 
> I see this as a help to quickly determine what I can do and cannot do 
> at a certain point of time. The goal is to remove the restrictions.
> 
> I am mostly interested in ARM and AVR32.
> There are a lot of packages, which needs an active porting effort to 
> build on the AVR32, since those packages explicitly state which 
> architectures are supported, and the AVR32 might not be listed.
> It would be very good if people would know up front what can be done at
> the moment.
> 
> On the ARM, the problem is different. 
> Packages, not involving direct access to architecure specific H/W
> (mostly PC stuff) should build, but for various reasons they do not.
> If people starting using buildroot had this help, then they could
> spend their time first building a working filesystem, and then select
> a specific, non-working package, to debug.
> Today endless hours can be spent on trying to build stuff which ends in
> a failure.
> If it is known not to build, it is good to communicate this to others.
> 
> 
> With this fix, it will be easy to grep package/*/Config.in 
> for a list of non-working packages, and then prioritize
> what to work on.
> 
> 
> 
> -- 
> Best Regards,
> Ulf Samuelsson
> 
> _______________________________________________
> buildroot mailing list
> buildroot at uclibc.org
> http://busybox.net/mailman/listinfo/buildroot
>

[Buildroot] Hiding non-buildable packages

[Yann E. MORIN]

Ulf, Bernhard and All,

On Wednesday 22 August 2007 22:24, Ulf Samuelsson wrote:
> If people starting using buildroot had this help, then they could
> spend their time first building a working filesystem, and then select
> a specific, non-working package, to debug.

For what it's worth, here's my point of view.

When _I_ start from scratch on something I have no previous knowledge of, then
I tend to build the smallest set I can. In this case, I'd disable everything in
buildroot, save building the toolchain.

Once I get that first step right, I start by adding things bit after bit. For
buildroot, BusyBox looks the most basic thing that can be added to a bare rootfs.

Again, once this is working, I would get more comfortable to try other things,
such as adding other tools, for example wireless tools (supposing I'd need that).

And so on till I get something that doesn't build, which I'd try to debug, submit
a patch or workaround, and resort to asking help if I can't solve it on my own.

What I'm trying to say is that someone starting with buildroot and expecting
to be able to build every and all packages shouldn't be surprised if something
goes amiss.

The rule is: divide and rule. Take pieces bit after bit while it works, and
you get a 'stable' basis to try a new feature. Bash this new feature until it
works, and start again.

On a personal experience, I have background that tells me what package to add
or not in a config. This is becaise I've had previous experience telling me
that such or such option is valis in such or such circumstances. Those are far
too complex to express in terms of Kconfig, I'm afraid.

> If it is known not to build, it is good to communicate this to others.

But it might build in some cases which are not your daily config.

My 2 euro-cents.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< ?_? >==-- ?------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
?------------------------------?-------?------------------?--------------------?

[Buildroot] Hiding non-buildable packages

[Ulf Samuelsson]

ons 2007-08-22 klockan 22:44 +0200 skrev Yann E. MORIN:
> Ulf, Bernhard and All,
> 
> On Wednesday 22 August 2007 22:24, Ulf Samuelsson wrote:
> > If people starting using buildroot had this help, then they could
> > spend their time first building a working filesystem, and then select
> > a specific, non-working package, to debug.
> 
> For what it's worth, here's my point of view.
> 
> When _I_ start from scratch on something I have no previous knowledge of, then
> I tend to build the smallest set I can. In this case, I'd disable everything in
> buildroot, save building the toolchain.
> 
> Once I get that first step right, I start by adding things bit after bit. For
> buildroot, BusyBox looks the most basic thing that can be added to a bare rootfs.
> 
> Again, once this is working, I would get more comfortable to try other things,
> such as adding other tools, for example wireless tools (supposing I'd need that).
> 
> And so on till I get something that doesn't build, which I'd try to debug, submit
> a patch or workaround, and resort to asking help if I can't solve it on my own.
> 
> What I'm trying to say is that someone starting with buildroot and expecting
> to be able to build every and all packages shouldn't be surprised if something
> goes amiss.
> 
> The rule is: divide and rule. Take pieces bit after bit while it works, and
> you get a 'stable' basis to try a new feature. Bash this new feature until it
> works, and start again.
> 
> On a personal experience, I have background that tells me what package to add
> or not in a config. This is becaise I've had previous experience telling me
> that such or such option is valis in such or such circumstances. Those are far
> too complex to express in terms of Kconfig, I'm afraid.
> 
> > If it is known not to build, it is good to communicate this to others.
> 
> But it might build in some cases which are not your daily configted, bn.

Granted, but do we need it to be perfect?, I think not!

I know that openssh does not work due to lack of 64 bit math.
I do not think it works for any architecture.
Why should every newbie try this out for themseleves,
only to find that it does not build...

Much better to hide it from the build, and then let the guy
which really wants it to reenable and debug it.


-- 
Best Regards,
Ulf Samuelsson

[Buildroot] Hiding non-buildable packages

[Bernhard Fischer]

On Wed, Aug 22, 2007 at 10:44:38PM +0200, Yann E. MORIN wrote:
>Ulf, Bernhard and All,
>
>On Wednesday 22 August 2007 22:24, Ulf Samuelsson wrote:
>> If people starting using buildroot had this help, then they could
>> spend their time first building a working filesystem, and then select
>> a specific, non-working package, to debug.
>
>For what it's worth, here's my point of view.
>
>When _I_ start from scratch on something I have no previous knowledge of, then
>I tend to build the smallest set I can. In this case, I'd disable everything in
>buildroot, save building the toolchain.
>
>Once I get that first step right, I start by adding things bit after bit. For
>buildroot, BusyBox looks the most basic thing that can be added to a bare rootfs.
>
>Again, once this is working, I would get more comfortable to try other things,
>such as adding other tools, for example wireless tools (supposing I'd need that).
>
>And so on till I get something that doesn't build, which I'd try to debug, submit
>a patch or workaround, and resort to asking help if I can't solve it on my own.
>
>What I'm trying to say is that someone starting with buildroot and expecting
>to be able to build every and all packages shouldn't be surprised if something
>goes amiss.
>
>The rule is: divide and rule. Take pieces bit after bit while it works, and

divide et impera, nod.

>you get a 'stable' basis to try a new feature. Bash this new feature until it
>works, and start again.
>
>On a personal experience, I have background that tells me what package to add
>or not in a config. This is becaise I've had previous experience telling me
>that such or such option is valis in such or such circumstances. Those are far
>too complex to express in terms of Kconfig, I'm afraid.
>
>> If it is known not to build, it is good to communicate this to others.
>
>But it might build in some cases which are not your daily config.

Normal dependencies that are hardware dependant are ok to add. Anything
else is not, since it just tried to work around bugs, which is
inacceptable.
>
>My 2 euro-cents.

It's not that we have a feature-rich and demanding default
configuration.. The default (including about all of busybox since that's
what's supposed to be tested alone uclibc by br) should build and of
course work correctly just fine, from my POV. Y(not Yann's)MMV

[Buildroot] Hiding non-buildable packages

[Yann E. MORIN]

On Wednesday 22 August 2007 22:59, Ulf Samuelsson wrote:
> Granted, but do we need it to be perfect?, I think not!
> I know that openssh does not work due to lack of 64 bit math.
> I do not think it works for any architecture.
> Why should every newbie try this out for themseleves,
> only to find that it does not build...
> Much better to hide it from the build, and then let the guy
> which really wants it to reenable and debug it.

The problem with your suggestion, if I may, is that it becomes quite complex
to maintain.

I'd rather have something much simpler, should we do want to add such options:

config MAYBE_BROKEN
    bool
    prompt "Show packages that might not build"
    default y
    help
      Show packages that, for a reason or another, might not build
      for some architecture(s).
      If you're lucky, it will build for yours. If not, you can try to
      fix it, and post your fix. :-)

config REALLY_BROKEN
    bool
    prompt "Show packages that won't build"
    default n
    help
      Some packages are so deeply broken that your help would be
      greatly appreciated should you post a patch solving the issue! :-)
      Warning: this option is not for the faint-hearted!

...

config BR2_PACKAGE_FOO
    bool
    prompt "foo (MAYBE BROKEN)"
    depends on MAYBE_BROKEN
    default n

config BR2_PACKAGE_BAR
    bool
    prompt "bar (BROKEN)"
    depends on REALLY_BROKEN
    default n


Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< ?_? >==-- ?------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
?------------------------------?-------?------------------?--------------------?

[Buildroot] Hiding non-buildable packages

[Bernhard Fischer]

On Wed, Aug 22, 2007 at 10:59:24PM +0200, Ulf Samuelsson wrote:

>Granted, but do we need it to be perfect?, I think not!
>
>I know that openssh does not work due to lack of 64 bit math.

You are - for the second time today - tempting me to just apply my
openssh update which i know may have trouble in some odd places, just
FYI..
>I do not think it works for any architecture.
>Why should every newbie try this out for themseleves,
>only to find that it does not build...

Since the default works fine, if somebody tries hard to shoot themselves
in the foot, why prevent them from doing so?
>
>Much better to hide it from the build, and then let the guy
>which really wants it to reenable and debug it.

No. That's not my approach.
Let defaults work, if you change anything, be prepared to keep all
parts.

[Buildroot] Hiding non-buildable packages

[Yann E. MORIN]

Brnhard, Ulf and All,

On Wednesday 22 August 2007 23:05, Bernhard Fischer wrote:
> divide et impera, nod.

I couldn't remember the latin words. Thank you!

> Normal dependencies that are hardware dependant are ok to add. Anything

Granted.

> else is not, since it just tried to work around bugs, which is
> inacceptable.

What about what I just posted back, should we want it?

> It's not that we have a feature-rich and demanding default
> configuration.. The default (including about all of busybox since that's
> what's supposed to be tested alone uclibc by br) should build and of
> course work correctly just fine, from my POV.

That's fine by me.

> Y(not Yann's)MMV 

Hehe! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< ?_? >==-- ?------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
?------------------------------?-------?------------------?--------------------?

[Buildroot] Hiding non-buildable packages

[Bernhard Fischer]

On Wed, Aug 22, 2007 at 11:17:35PM +0200, Yann E. MORIN wrote:
>On Wednesday 22 August 2007 22:59, Ulf Samuelsson wrote:
>> Granted, but do we need it to be perfect?, I think not!
>> I know that openssh does not work due to lack of 64 bit math.
>> I do not think it works for any architecture.
>> Why should every newbie try this out for themseleves,
>> only to find that it does not build...
>> Much better to hide it from the build, and then let the guy
>> which really wants it to reenable and debug it.
>
>The problem with your suggestion, if I may, is that it becomes quite complex
>to maintain.

Yes, this is nonsense and we will simply not do that but strive to
maximize our working set.

Ulf, that's exactly the reason why -- up until recently -- didn't add
any random package some single person wanted to use. Keep in mind that
*you* have to _actively_ maintain the packages you add, or at least try
to, to the best of your knowledge/time.
>
>I'd rather have something much simpler, should we do want to add such options:
>
>config MAYBE_BROKEN
>    bool

false
Stiff should work, ideally. The default _has_ to work, about anything
else could be broken anytime. If it's broken, then fix it, if you need
it, else turn it off. That simple.

[Buildroot] Hiding non-buildable packages

[Bernhard Fischer]

On Wed, Aug 22, 2007 at 11:05:23PM +0200, Bernhard Fischer wrote:

>It's not that we have a feature-rich and demanding default
>configuration.. The default (including about all of busybox since that's
>what's supposed to be tested alone uclibc by br) should build and of

s/alone/along/;# even. BAH!

>course work correctly just fine, from my POV. Y(not Yann's)MMV

[Buildroot] Hiding non-buildable packages

[Yann E. MORIN]

On Wednesday 22 August 2007 23:21, Yann E. MORIN wrote:
> Brnhard, Ulf and All,
s/Brnhard/Bernhard/; # Sorry... Time to go to bed, I guess... :-/

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< ?_? >==-- ?------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
?------------------------------?-------?------------------?--------------------?

[Buildroot] Hiding non-buildable packages

[Ulf Samuelsson]

ons 2007-08-22 klockan 23:17 +0200 skrev Yann E. MORIN:
> On Wednesday 22 August 2007 22:59, Ulf Samuelsson wrote:
> > Granted, but do we need it to be perfect?, I think not!
> > I know that openssh does not work due to lack of 64 bit math.
> > I do not think it works for any architecture.
> > Why should every newbie try this out for themseleves,
> > only to find that it does not build...
> > Much better to hide it from the build, and then let the guy
> > which really wants it to reenable and debug it.
> 
> The problem with your suggestion, if I may, is that it becomes quite complex
> to maintain.
> 

No, I think you want for each target you want to have a dependency on 
a negative variable which might or might not be defined.

	depends on !HIDE_PACKAGE_<package>

is added *once* for each packet.


"package/<package>/Config.in" will not have to be touched ever again.
(At least for purpose of maintaining list of buggy package)
You can then hide this packet for several reasons


Then you have an additional file which contains all the dependencies.

config	REALLY_BROKEN_ARM
	depends on BR2_arm && BR2_REALLY_BROKEN
	select HIDE_PACKAGE_1
	select ...
	select ...

config	MAYBE_BROKEN_ARM
	depends on BR2_arm && BR2_MAYBE_BROKEN
	select HIDE_PACKAGE_17
	select ...
	select ...

config	REALLY_BROKEN_X866
	depends on BR2_arm && BR2_REALLY_BROKEN
	select ...
	select ...
	select ...


All your maintenance is then performed in this file.
If you want to move a package from suspicous to really broken
you do that easily within an editor.


> I'd rather have something much simpler, should we do want to add such options:
> 
> config MAYBE_BROKEN
>     bool
>     prompt "Show packages that might not build"
>     default y
>     help
>       Show packages that, for a reason or another, might not build
>       for some architecture(s).
>       If you're lucky, it will build for yours. If not, you can try to
>       fix it, and post your fix. :-)
> 
> config REALLY_BROKEN
>     bool
>     prompt "Show packages that won't build"
>     default n
>     help
>       Some packages are so deeply broken that your help would be
>       greatly appreciated should you post a patch solving the issue! :-)
>       Warning: this option is not for the faint-hearted!
> 
> ...
> 
> config BR2_PACKAGE_FOO
>     bool
>     prompt "foo (MAYBE BROKEN)"
>     depends on MAYBE_BROKEN
>     default n
> 
> config BR2_PACKAGE_BAR
>     bool
>     prompt "bar (BROKEN)"
>     depends on REALLY_BROKEN
>     default n
> 
> 
> Regards,
> Yann E. MORIN.
> 
-- 
Best Regards,
Ulf Samuelsson

[Buildroot] Hiding non-buildable packages

[Ulf Samuelsson]

ons 2007-08-22 klockan 23:20 +0200 skrev Bernhard Fischer:
> On Wed, Aug 22, 2007 at 10:59:24PM +0200, Ulf Samuelsson wrote:
> 
> >Granted, but do we need it to be perfect?, I think not!
> >
> >I know that openssh does not work due to lack of 64 bit math.
> 
> You are - for the second time today - tempting me to just apply my
> openssh update which i know may have trouble in some odd places, just
> FYI..

Go ahead, since it is permanently broken.

Implementing the 64 bit math routines in uClibc as an option
depending on if openSSH is built would be the proper thing to do.

- Or put the 64 bit math in the openSSH tree in a lib,

- Or change the computation to 32 bit math.


> >I do not think it works for any architecture.
> >Why should every newbie try this out for themseleves,
> >only to find that it does not build...
> 
> Since the default works fine, if somebody tries hard to shoot themselves
> in the foot, why prevent them from doing so?
> >
> >Much better to hide it from the build, and then let the guy
> >which really wants it to reenable and debug it.
> 
> No. That's not my approach.
> Let defaults work, if you change anything, be prepared to keep all
> parts.

I am not suggesting that the default should change.
The default should still be to show all packages,
but you can, set an option which hides broken packages


-- 
Best Regards,
Ulf Samuelsson          mail:   ulf at atmel.com
Atmel Nordic AB
Box 2033, 174 52 Sundbyberg
Kavalleriv?gen 24, 174 58 Sundbyberg
Sweden
Tel:    +46 8 441 54 22 GSM:    +46 706 224457