[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Yann E. MORIN]

Hello All !

This is a Request For Comments about the now-infamous dilemn between
"depends on" vs. "select", and a suggestion to work it around.

As I am adding some packages (soon to be pushed!), I stumbled upon a few
cases where I could not select another package because it had dependencies
my package does not have.

And as we all know, doing so is A Very Bad Idea (TM).

So I decided to depend on this package, just printing a message in case it's
not available. Sad-and-dull life, but life nonetheless...

But as an afterthought, I thought about a /solution/ to this issue. It's
best explained with a little bit of code, so consider the following:

  config PKG_LIBFOO_AVAILABLE
    def_bool y
    depends on ARM && THREADS && !LFS

  config PKG_LIBFOO
    bool "libfoo"
    depends on PKG_LIBFOO_AVAILABLE

  comments "libfoo requires ARM and threads, but not Large files"
    depends on !PKG_LIBFOO_AVAILABLE

That is, move all the package's dependencies to their own symbol, and
have the package depends on this symbol.

Although the gain is not explicit for the package libfoo by itself, it
becomes much more evident when another package depends on libfoo:

  config PKG_BAR
    bool "bar"
    depends on PKG_LIBFOO_AVAILABLE
    select PKG_LIBFOO

This way, it becomes extremely easy to inherit the dependencies of libfoo
in bar, and still we can select it. If the dependencies of libfoo were to
change, there would be a single place where to update them.

(And yes, we could/should also have a PKG_BAR_AVAILABLE.)

This also allows for more involved constructs (which is the actual use-case
I'm in need for my package):

  config PKG_BAR
    bool "bar"

  config PKG_BAR_OPT
    bool "bar option"
    depends on PKG_BAR
    depends on PKG_LIBFOO_AVAILABLE
    select PKG_LIBFOO

(I have tested both constructs with the current kconfig from 3.6-rc1, and
it behaves as expected.)

*And* it has no impact whatsoever on the package build infrastructure.

Of course, this does not have a zero-impact on how to write packages'
Config.in files, and there is no easy way to convert existing packages.
But, as for the package infrastructure, this could be a step-by-step
process by which packages would be converted (enhanced!) one after the
other.

Another disadvantage is that it wanders a bit away from the current
simplicity of writing a single package; complexity that is regained
later-on when new packages will use it.

What are your thoughts on this?
Where's the flaw in my reasoning? ;-)
Am I rambling on incoherently? ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Thomas Petazzoni]

Hello!

Le Tue, 7 Aug 2012 22:52:17 +0200,
"Yann E. MORIN" <yann.morin.1998@free.fr> a ?crit :

> This is a Request For Comments about the now-infamous dilemn between
> "depends on" vs. "select", and a suggestion to work it around.
> 
> As I am adding some packages (soon to be pushed!), I stumbled upon a
> few cases where I could not select another package because it had
> dependencies my package does not have.
> 
> And as we all know, doing so is A Very Bad Idea (TM).
> 
> So I decided to depend on this package, just printing a message in
> case it's not available. Sad-and-dull life, but life nonetheless...
> 
> But as an afterthought, I thought about a /solution/ to this issue.
> It's best explained with a little bit of code, so consider the
> following:
> 
>   config PKG_LIBFOO_AVAILABLE
>     def_bool y
>     depends on ARM && THREADS && !LFS
> 
>   config PKG_LIBFOO
>     bool "libfoo"
>     depends on PKG_LIBFOO_AVAILABLE
> 
>   comments "libfoo requires ARM and threads, but not Large files"
>     depends on !PKG_LIBFOO_AVAILABLE
> 
> That is, move all the package's dependencies to their own symbol, and
> have the package depends on this symbol.
> 
> Although the gain is not explicit for the package libfoo by itself, it
> becomes much more evident when another package depends on libfoo:
> 
>   config PKG_BAR
>     bool "bar"
>     depends on PKG_LIBFOO_AVAILABLE
>     select PKG_LIBFOO
> 
> This way, it becomes extremely easy to inherit the dependencies of
> libfoo in bar, and still we can select it. If the dependencies of
> libfoo were to change, there would be a single place where to update
> them.
> 
> (And yes, we could/should also have a PKG_BAR_AVAILABLE.)
> 
> This also allows for more involved constructs (which is the actual
> use-case I'm in need for my package):
> 
>   config PKG_BAR
>     bool "bar"
> 
>   config PKG_BAR_OPT
>     bool "bar option"
>     depends on PKG_BAR
>     depends on PKG_LIBFOO_AVAILABLE
>     select PKG_LIBFOO
> 
> (I have tested both constructs with the current kconfig from 3.6-rc1,
> and it behaves as expected.)

Thanks a lot for thinking about this. So, the idea is that every time
there is a "select", we add a corresponding "depends on", and that
since there is inheritance of "depends on", we will not have to
recursively add all the "depends on" on toolchain options on all the
reverse dependencies of a package that need such toolchain options.

Am I understanding your proposal correctly?

> *And* it has no impact whatsoever on the package build infrastructure.
> 
> Of course, this does not have a zero-impact on how to write packages'
> Config.in files, and there is no easy way to convert existing
> packages. But, as for the package infrastructure, this could be a
> step-by-step process by which packages would be converted (enhanced!)
> one after the other.
> 
> Another disadvantage is that it wanders a bit away from the current
> simplicity of writing a single package; complexity that is regained
> later-on when new packages will use it.
> 
> What are your thoughts on this?
> Where's the flaw in my reasoning? ;-)
> Am I rambling on incoherently? ;-)

On my side, I would need to experiment a bit with this on some
real-world example, but it sounds like a nice path to investigate, for
sure.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Yann E. MORIN]

Thomas, All,

On Thursday 09 August 2012 11:40:01 Thomas Petazzoni wrote:
> Le Tue, 7 Aug 2012 22:52:17 +0200,
> "Yann E. MORIN" <yann.morin.1998@free.fr> a ?crit :
> > This is a Request For Comments about the now-infamous dilemn between
> > "depends on" vs. "select", and a suggestion to work it around.
[--SNIP--]
> Thanks a lot for thinking about this. So, the idea is that every time
> there is a "select", we add a corresponding "depends on", and that
> since there is inheritance of "depends on", we will not have to
> recursively add all the "depends on" on toolchain options on all the
> reverse dependencies of a package that need such toolchain options.
> 
> Am I understanding your proposal correctly?

Basically, yes. I was only thinking about inter-package dependencies, but
it can be of course applied to other options.

Where this would usefull:
 - a package depends on some generic options: toolchain, architecture...
 - a package depends on another package.

Adding the intermediate _AVAILABLE symbol propagates the dependency list,
so all a package (or a sub-option of a package) needs to know is wether
that other package it depends on is available, so it can select it.

But there is a limitation: should we also use that mechanism for package
options? Eg:

  config PKG_FOO_AVAIL
    def_bool y
    depends on WCHAR

  config PKG_FOO
    bool "foo"
    depends on PKG_FOO_AVAIL

  if PKG_FOO

  config PKG_FOO_OPT_AVAIL
    def_bool y
    depends on ARM

  config PKG_FOO_OPT
    bool "opt"
    depends on PKG_FOO_OPT_AVAIL

  endif # PKG_FOO

  config PKG_BAR
    bool "bar"
    depends on PKG_FOO_AVAIL && PKG_FOO_OPT_AVAIL
    select PKG_FOO
    select PKG_FOO_OPT

Going this route can lead to over-complicated and over-cluterred Config.in
files. We have to set a limit to where we want to go.

> On my side, I would need to experiment a bit with this on some
> real-world example, but it sounds like a nice path to investigate, for
> sure.

I'm finishing my new packages that I'll push soon (this WE?) because it
takes a bit of time testing all the combinations.

When those are pushed, I'll try to add this _AVAILABLE stuff to the packages
I added and their depndencies to see if it plays nicely.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Thomas Petazzoni]

Le Thu, 9 Aug 2012 11:58:19 +0200,
"Yann E. MORIN" <yann.morin.1998@free.fr> a ?crit :

> Going this route can lead to over-complicated and over-cluterred
> Config.in files. We have to set a limit to where we want to go.

Right. I guess we can limit this do the package that are being
select'ed by others. Generally, this means the libraries and a few
other packages.

I must admit I still see this as a workaround for a
deficiency/limitation of kconfig. It's true that it will make our
package Config.in files a bit less obvious to understand.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Alex Bradbury]

On 9 August 2012 12:30, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Thu, 9 Aug 2012 11:58:19 +0200,
> I must admit I still see this as a workaround for a
> deficiency/limitation of kconfig. It's true that it will make our
> package Config.in files a bit less obvious to understand.

Is that "I'd prefer to see a patch to add this functionality to
buildroot's kconfig?".

Alex

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Yann E. MORIN]

Alex, Thomas, All,

On Thursday 09 August 2012 13:57:02 Alex Bradbury wrote:
> On 9 August 2012 12:30, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Le Thu, 9 Aug 2012 11:58:19 +0200,
> > I must admit I still see this as a workaround for a
> > deficiency/limitation of kconfig. It's true that it will make our
> > package Config.in files a bit less obvious to understand.
> 
> Is that "I'd prefer to see a patch to add this functionality to
> buildroot's kconfig?".

I think that kconfig changes should go upstream (ie. the linux kernel) first.

Such a change will probably be very intrusive; and there are very
kconfig-knowledgeable persons on the linux-kbuild ML that can rhelp eview
such an impacting change.

Of course, nothing prevents one from testing that change against buildroot! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |   ^                |
| --==< O_o >==-- '------------.-------:  X  AGAINST      |  /e\  There is no  |
| http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL    |  """  conspiracy.  |
'------------------------------'-------'------------------'--------------------'

[Buildroot] [RFC] "select" vs. "depends on" : a proposal for a workaround

[Thomas Petazzoni]

Le Thu, 9 Aug 2012 12:57:02 +0100,
Alex Bradbury <asb@asbradbury.org> a ?crit :

> Is that "I'd prefer to see a patch to add this functionality to
> buildroot's kconfig?".

Yes.

And of course, as Yann said, a functionality added to the mainline
kconfig so that we don't have to carry this (most likely complicated)
modification as a patch against the official kconfig.

I started looking at this a bit, but the kconfig code is not that
simple, and so horribly under-documented.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com