[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Carlos Santos]

The plan for the future is:

 * All host packages have a Config.in.host option.

 * The host packages that are only build dependencies of other packages
   have a blind Config.in.host option

 * The host packages that are useful by themselves continue to have a
   visible Config.in.host option.

host-waf gets a blind Config.in.host, because it exists only to build
Waf-based packages that set <PKG>_NEEDS_EXTERNAL_WAF to YES.

A help text is included to document the package, only, since it is not
shown in the configuration menu.

Signed-off-by: Carlos Santos <casantos@datacom.com.br>
---
Changes v1->v2:
  - Explain the motivation in the commit message
  - Make the Config.in.host blind
Changes v2->v3:
  - Removes "quoting Thomas Petazzoni" from the commit messege, since
    it's not a personal opinion but a consensus among project members.
---
 package/Config.in.host     | 1 +
 package/waf/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/waf/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 16b474fc9d..3644436fe3 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -64,6 +64,7 @@ menu "Host utilities"
 	source "package/uboot-tools/Config.in.host"
 	source "package/util-linux/Config.in.host"
 	source "package/vboot-utils/Config.in.host"
+	source "package/waf/Config.in.host"
 	source "package/xorriso/Config.in.host"
 	source "package/zip/Config.in.host"
 	source "package/zstd/Config.in.host"
diff --git a/package/waf/Config.in.host b/package/waf/Config.in.host
new file mode 100644
index 0000000000..81a5c5cc27
--- /dev/null
+++ b/package/waf/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_WAF
+	bool
+	help
+	  WAF, the meta build system
+
+	  https://waf.io/
-- 
2.20.1

[Buildroot] [PATCH v3 2/3] doc/manual: document the waf packages may need to select host-waf

[Carlos Santos]

Now that waf has a blind Config.in.host we can require that packages
setting <PKG>_NEEDS_EXTERNAL_WAF to YES slso select host-waf.

Signed-off-by: Carlos Santos <casantos@datacom.com.br>
---
 docs/manual/adding-packages-waf.txt | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/docs/manual/adding-packages-waf.txt b/docs/manual/adding-packages-waf.txt
index 43b557c070..f38c3a6f55 100644
--- a/docs/manual/adding-packages-waf.txt
+++ b/docs/manual/adding-packages-waf.txt
@@ -64,7 +64,10 @@ also be defined.
   Buildroot to use the bundled +waf+ executable. If set to +NO+, the
   default, then Buildroot will use the waf executable provided in the
   package source tree; if set to +YES+, then Buidlroot will download,
-  install waf as a host tool and use it to build the package.
+  install waf as a host tool and use it to build the package. Packages
+  that need the host waf tool must select +BR2_PACKAGE_HOST_WAF+ in
+  their Config.in. It is not necessary to add host-waf to
+  +LIBFOO_DEPENDENCIES+, since this is done atomatically.
 
 * +LIBFOO_WAF_OPTS+, to specify additional options to pass to the
   +waf+ script at every step of the package build process: configure,
-- 
2.20.1

[Buildroot] [PATCH v3 2/3] doc/manual: document the waf packages may need to select host-waf

[Thomas Petazzoni]

Hello Carlos,

Minor nit: title should use the prefix docs/manual.

On Sun, 23 Dec 2018 15:19:49 -0200, Carlos Santos wrote:
> Now that waf has a blind Config.in.host we can require that packages
> setting <PKG>_NEEDS_EXTERNAL_WAF to YES slso select host-waf.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.com.br>
> ---
>  docs/manual/adding-packages-waf.txt | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/docs/manual/adding-packages-waf.txt b/docs/manual/adding-packages-waf.txt
> index 43b557c070..f38c3a6f55 100644
> --- a/docs/manual/adding-packages-waf.txt
> +++ b/docs/manual/adding-packages-waf.txt
> @@ -64,7 +64,10 @@ also be defined.
>    Buildroot to use the bundled +waf+ executable. If set to +NO+, the
>    default, then Buildroot will use the waf executable provided in the
>    package source tree; if set to +YES+, then Buidlroot will download,
> -  install waf as a host tool and use it to build the package.
> +  install waf as a host tool and use it to build the package. Packages
> +  that need the host waf tool must select +BR2_PACKAGE_HOST_WAF+ in

Perhaps: "Packages in this situation must select..."

> +  their Config.in. It is not necessary to add host-waf to
> +  +LIBFOO_DEPENDENCIES+, since this is done atomatically.

atomatically -> automatically.

(Note: I would have fixed these minors nits myself when applying, but I
sent some other questions/comments on PATCH 1/3 anyway.)

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 3/3] package/mpv: selec host-waf

[Carlos Santos]

It's required, since MPV_NEEDS_EXTERNAL_WAF is set to YES in mpv.mk.

Signed-off-by: Carlos Santos <casantos@datacom.com.br>
---
 package/mpv/Config.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/mpv/Config.in b/package/mpv/Config.in
index 69810038f1..ed9662759d 100644
--- a/package/mpv/Config.in
+++ b/package/mpv/Config.in
@@ -16,6 +16,7 @@ config BR2_PACKAGE_MPV
 	select BR2_PACKAGE_XLIB_LIBXINERAMA if BR2_PACKAGE_XORG7
 	select BR2_PACKAGE_XLIB_LIBXSCRNSAVER if BR2_PACKAGE_XORG7
 	select BR2_PACKAGE_XLIB_LIBXRANDR if BR2_PACKAGE_XORG7
+	select BR2_PACKAGE_HOST_WAF
 	help
 	  mpv is a fork of mplayer2 and MPlayer. It shares some features
 	  with the former projects while introducing many more.
-- 
2.20.1

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Thomas Petazzoni]

Hello Carlos,

On Sun, 23 Dec 2018 15:19:48 -0200, Carlos Santos wrote:
> The plan for the future is:
> 
>  * All host packages have a Config.in.host option.
> 
>  * The host packages that are only build dependencies of other packages
>    have a blind Config.in.host option
> 
>  * The host packages that are useful by themselves continue to have a
>    visible Config.in.host option.
> 
> host-waf gets a blind Config.in.host, because it exists only to build
> Waf-based packages that set <PKG>_NEEDS_EXTERNAL_WAF to YES.
> 
> A help text is included to document the package, only, since it is not
> shown in the configuration menu.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.com.br>
> ---
> Changes v1->v2:
>   - Explain the motivation in the commit message

In fact the commit message doesn't explain the actual motivation. What
prompted you to add a blind Config.in.host option for waf
specifically ? Is this just a part of "let's start adding blind option
for all host packages", or is there a specific issue with host-waf that
this blind Config.in.host option will allow to solve ?

I.e: I am all in favor of this change, but I'm just curious to
understand why you did it in the first place.

> diff --git a/package/Config.in.host b/package/Config.in.host
> index 16b474fc9d..3644436fe3 100644
> --- a/package/Config.in.host
> +++ b/package/Config.in.host
> @@ -64,6 +64,7 @@ menu "Host utilities"
>  	source "package/uboot-tools/Config.in.host"
>  	source "package/util-linux/Config.in.host"
>  	source "package/vboot-utils/Config.in.host"
> +	source "package/waf/Config.in.host"
>  	source "package/xorriso/Config.in.host"
>  	source "package/zip/Config.in.host"
>  	source "package/zstd/Config.in.host"

I'm not sure it's really nice to include all the host packages
Config.in.host files inside the "Host utilities" menu. In practice,
it's a blind option so it doesn't matter much, but it will clutter a
lot this list, making it difficult to see which one is there to be
really visible in menuconfig, and which one is here just so that the
kconfig machinery knows about all those blind options.

I think I would prefer a new list, at the bottom of
package/Config.in.host, outside of the Host utilities menu.

Let's see what Arnout/Peter/Yann have to say about this.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Yann E. MORIN]

Thomas, Carlos, All,

On 2018-12-26 22:30 +0100, Thomas Petazzoni spake thusly:
> On Sun, 23 Dec 2018 15:19:48 -0200, Carlos Santos wrote:
> > The plan for the future is:
> > 
> >  * All host packages have a Config.in.host option.
> > 
> >  * The host packages that are only build dependencies of other packages
> >    have a blind Config.in.host option
> > 
> >  * The host packages that are useful by themselves continue to have a
> >    visible Config.in.host option.
> > 
> > host-waf gets a blind Config.in.host, because it exists only to build
> > Waf-based packages that set <PKG>_NEEDS_EXTERNAL_WAF to YES.
> > 
> > A help text is included to document the package, only, since it is not
> > shown in the configuration menu.
> > 
> > Signed-off-by: Carlos Santos <casantos@datacom.com.br>
> > ---
> > Changes v1->v2:
> >   - Explain the motivation in the commit message
> 
> In fact the commit message doesn't explain the actual motivation. What
> prompted you to add a blind Config.in.host option for waf
> specifically ? Is this just a part of "let's start adding blind option
> for all host packages", or is there a specific issue with host-waf that
> this blind Config.in.host option will allow to solve ?
> 
> I.e: I am all in favor of this change, but I'm just curious to
> understand why you did it in the first place.

Well, I am not really happy with that, though: do we really plan on
having packages really select all the host tools they need?

If so, do we really envision autotools-based packages selecting
host-autoconf, host-automake, host-libtool? And then packages that use
host-pkgconf you should also select it...

Also, what about host-cmake, which is conditionally built, but for which
we do not have the info in kconfig? (well, we can argue we'd have to do
like we do for host-gcc, but still). Oh, and host-tar, host-flex,
host-bison, and so on... :-/

So, no, I'm not happy with that direction...

    config BR2_PACKAGE_FOO
        bool "foo"
        select BR2_PACKAGE_HOST_AUTOCONF
        select BR2_PACKAGE_HOST_AUTOMAKE
        select BR2_PACKAGE_HOST_LIBTOOL
        select BR2_PACKAGE_MAYBE_HOST_TAR
        select BR2_PACKAGE_MAYBE_HOST_FLEX_FOR_KCONFIG
        select BR2_PACKAGE_MAYBE_HOST_BISON_FOR_KCONFIG
        select BR2_PACKAGE_HOST_PKGCONF

Unles we're planning on hiding that away into meta-config, like:

    config BR2_PACKAGE_FOO
        bool "foo"
        select BR2_AUTOTOOLS_PACKAGE # mimick $(eval $(autotools-package))
        select BR2_PACKAGE_HOST_PKGCONF_BECAUSE_IT_S_NOT_MANDATORY

And still, the optionally-required host packages like tar, flex et al.
are not covered...

Meh... :-(

> > diff --git a/package/Config.in.host b/package/Config.in.host
> > index 16b474fc9d..3644436fe3 100644
> > --- a/package/Config.in.host
> > +++ b/package/Config.in.host
> > @@ -64,6 +64,7 @@ menu "Host utilities"
> >  	source "package/uboot-tools/Config.in.host"
> >  	source "package/util-linux/Config.in.host"
> >  	source "package/vboot-utils/Config.in.host"
> > +	source "package/waf/Config.in.host"
> >  	source "package/xorriso/Config.in.host"
> >  	source "package/zip/Config.in.host"
> >  	source "package/zstd/Config.in.host"
> 
> I'm not sure it's really nice to include all the host packages
> Config.in.host files inside the "Host utilities" menu. In practice,
> it's a blind option so it doesn't matter much, but it will clutter a
> lot this list, making it difficult to see which one is there to be
> really visible in menuconfig, and which one is here just so that the
> kconfig machinery knows about all those blind options.
> 
> I think I would prefer a new list, at the bottom of
> package/Config.in.host, outside of the Host utilities menu.
> 
> Let's see what Arnout/Peter/Yann have to say about this.

If we have to have it, then indeed, let's at least move it away of the
user-selectable ones, into its own section, yes.

Regards,
Yann E. MORIN.

> Best regards,
> 
> Thomas
> -- 
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Thomas Petazzoni]

Hello,

+Arnout in Cc. Arnout, there's some discussion on one of your favorite
topic: Config.in.host options for all packages. Read on below.

On Wed, 26 Dec 2018 22:55:21 +0100, Yann E. MORIN wrote:

> > I.e: I am all in favor of this change, but I'm just curious to
> > understand why you did it in the first place.  
> 
> Well, I am not really happy with that, though: do we really plan on
> having packages really select all the host tools they need?
> 
> If so, do we really envision autotools-based packages selecting
> host-autoconf, host-automake, host-libtool? And then packages that use
> host-pkgconf you should also select it...
> 
> Also, what about host-cmake, which is conditionally built, but for which
> we do not have the info in kconfig? (well, we can argue we'd have to do
> like we do for host-gcc, but still). Oh, and host-tar, host-flex,
> host-bison, and so on... :-/

Meh, I hadn't thought of conditional packages like host-tar, host-cmake
and so on.

> So, no, I'm not happy with that direction...
> 
>     config BR2_PACKAGE_FOO
>         bool "foo"
>         select BR2_PACKAGE_HOST_AUTOCONF
>         select BR2_PACKAGE_HOST_AUTOMAKE
>         select BR2_PACKAGE_HOST_LIBTOOL
>         select BR2_PACKAGE_MAYBE_HOST_TAR
>         select BR2_PACKAGE_MAYBE_HOST_FLEX_FOR_KCONFIG
>         select BR2_PACKAGE_MAYBE_HOST_BISON_FOR_KCONFIG
>         select BR2_PACKAGE_HOST_PKGCONF
> 
> Unles we're planning on hiding that away into meta-config, like:
> 
>     config BR2_PACKAGE_FOO
>         bool "foo"
>         select BR2_AUTOTOOLS_PACKAGE # mimick $(eval $(autotools-package))
>         select BR2_PACKAGE_HOST_PKGCONF_BECAUSE_IT_S_NOT_MANDATORY
> 
> And still, the optionally-required host packages like tar, flex et al.
> are not covered...
> 
> Meh... :-(

Indeed, I understand the "Meh" here. I hadn't really realized what it
would mean to have Config.in.host options for all packages, and
properly selected by all its users.

But still, there are a number of cases where it would really help, so
that a given host package can be aware that another host package has
been built with a given feature (or not). Or precisely to force that a
certain host package is built with a given option. For example, in
host-python, we had situations where only a given package needed
host-python to be built with FOO support, and since we don't have any
BR2_PACKAGE_HOST_PYTHON_FOO option, our only choice was to
unconditionally enable FOO support in host-python, adding build time to
everyone, even if FOO in host-python might only be needed for one
obscure package.

It seems like we don't have a very conclusive decision on this topic at
this point.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Yann E. MORIN]

Thomas, All,

On 2019-01-03 22:50 +0100, Thomas Petazzoni spake thusly:
> On Wed, 26 Dec 2018 22:55:21 +0100, Yann E. MORIN wrote:
> > Well, I am not really happy with that, though: do we really plan on
> > having packages really select all the host tools they need?
> > 
> > If so, do we really envision autotools-based packages selecting
> > host-autoconf, host-automake, host-libtool? And then packages that use
> > host-pkgconf you should also select it...
> > 
> > Also, what about host-cmake, which is conditionally built, but for which
> > we do not have the info in kconfig? (well, we can argue we'd have to do
> > like we do for host-gcc, but still). Oh, and host-tar, host-flex,
> > host-bison, and so on... :-/
> 
> Meh, I hadn't thought of conditional packages like host-tar, host-cmake
> and so on.
> 
> > So, no, I'm not happy with that direction...
> > 
> >     config BR2_PACKAGE_FOO
> >         bool "foo"
> >         select BR2_PACKAGE_HOST_AUTOCONF
> >         select BR2_PACKAGE_HOST_AUTOMAKE
> >         select BR2_PACKAGE_HOST_LIBTOOL
> >         select BR2_PACKAGE_MAYBE_HOST_TAR
> >         select BR2_PACKAGE_MAYBE_HOST_FLEX_FOR_KCONFIG
> >         select BR2_PACKAGE_MAYBE_HOST_BISON_FOR_KCONFIG
> >         select BR2_PACKAGE_HOST_PKGCONF
> > 
> > Unles we're planning on hiding that away into meta-config, like:
> > 
> >     config BR2_PACKAGE_FOO
> >         bool "foo"
> >         select BR2_AUTOTOOLS_PACKAGE # mimick $(eval $(autotools-package))
> >         select BR2_PACKAGE_HOST_PKGCONF_BECAUSE_IT_S_NOT_MANDATORY
> > 
> > And still, the optionally-required host packages like tar, flex et al.
> > are not covered...
> > 
> > Meh... :-(
> 
> Indeed, I understand the "Meh" here. I hadn't really realized what it
> would mean to have Config.in.host options for all packages, and
> properly selected by all its users.

To be fait, I was initially also in favour of adding Config.in.host
options for all host packages. But this waf patch made me change my
mind, becasue of the above...

> But still, there are a number of cases where it would really help, so
> that a given host package can be aware that another host package has
> been built with a given feature (or not). Or precisely to force that a
> certain host package is built with a given option. For example, in
> host-python, we had situations where only a given package needed
> host-python to be built with FOO support, and since we don't have any
> BR2_PACKAGE_HOST_PYTHON_FOO option, our only choice was to
> unconditionally enable FOO support in host-python, adding build time to
> everyone, even if FOO in host-python might only be needed for one
> obscure package.

In this case, maybe we could simply depart from the rule, and just have
python/Config.in.host contain just:

    # Select this if you need host-python to support 'stuff'
    config BR2_NEEDS_HOST_PYTHON_WITH_STUFF
        bool

And then, python/python.mk would have:

    ifeq ($(BR2_NEEDS_HOST_PYTHON_WITH_STUFF),y)
    HOST_PYTHON_CONF_OPTS += --with-stuff
    else
    HOST_PYTHON_CONF_OPTS += --without-stuff
    endif

Regards,
Yann E. MORIN.

> It seems like we don't have a very conclusive decision on this topic at
> this point.


-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Yann E. MORIN]

All,

On 2019-01-03 23:06 +0100, Yann E. MORIN spake thusly:
> To be fait,

*fair

rgd.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Carlos Santos]

> From: "Yann Morin" <yann.morin.1998@free.fr>
> To: "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
> Cc: "Peter Korsgaard" <peter@korsgaard.com>, "Mahyar Koshkouei" <mahyar.koshkouei@gmail.com>, "Thomas De Schampheleire"
> <thomas.de_schampheleire@nokia.com>, "buildroot" <buildroot@buildroot.org>
> Sent: Quinta-feira, 3 de janeiro de 2019 20:06:52
> Subject: Re: [Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

> Thomas, All,
> 
> On 2019-01-03 22:50 +0100, Thomas Petazzoni spake thusly:
>> On Wed, 26 Dec 2018 22:55:21 +0100, Yann E. MORIN wrote:
>> > Well, I am not really happy with that, though: do we really plan on
>> > having packages really select all the host tools they need?
>> > 
>> > If so, do we really envision autotools-based packages selecting
>> > host-autoconf, host-automake, host-libtool? And then packages that use
>> > host-pkgconf you should also select it...
>> > 
>> > Also, what about host-cmake, which is conditionally built, but for which
>> > we do not have the info in kconfig? (well, we can argue we'd have to do
>> > like we do for host-gcc, but still). Oh, and host-tar, host-flex,
>> > host-bison, and so on... :-/
>> 
>> Meh, I hadn't thought of conditional packages like host-tar, host-cmake
>> and so on.
>> 
>> > So, no, I'm not happy with that direction...
>> > 
>> >     config BR2_PACKAGE_FOO
>> >         bool "foo"
>> >         select BR2_PACKAGE_HOST_AUTOCONF
>> >         select BR2_PACKAGE_HOST_AUTOMAKE
>> >         select BR2_PACKAGE_HOST_LIBTOOL
>> >         select BR2_PACKAGE_MAYBE_HOST_TAR
>> >         select BR2_PACKAGE_MAYBE_HOST_FLEX_FOR_KCONFIG
>> >         select BR2_PACKAGE_MAYBE_HOST_BISON_FOR_KCONFIG
>> >         select BR2_PACKAGE_HOST_PKGCONF
>> > 
>> > Unles we're planning on hiding that away into meta-config, like:
>> > 
>> >     config BR2_PACKAGE_FOO
>> >         bool "foo"
>> >         select BR2_AUTOTOOLS_PACKAGE # mimick $(eval $(autotools-package))
>> >         select BR2_PACKAGE_HOST_PKGCONF_BECAUSE_IT_S_NOT_MANDATORY
>> > 
>> > And still, the optionally-required host packages like tar, flex et al.
>> > are not covered...
>> > 
>> > Meh... :-(
>> 
>> Indeed, I understand the "Meh" here. I hadn't really realized what it
>> would mean to have Config.in.host options for all packages, and
>> properly selected by all its users.
> 
> To be fait, I was initially also in favour of adding Config.in.host
> options for all host packages. But this waf patch made me change my
> mind, becasue of the above...
> 
>> But still, there are a number of cases where it would really help, so
>> that a given host package can be aware that another host package has
>> been built with a given feature (or not). Or precisely to force that a
>> certain host package is built with a given option. For example, in
>> host-python, we had situations where only a given package needed
>> host-python to be built with FOO support, and since we don't have any
>> BR2_PACKAGE_HOST_PYTHON_FOO option, our only choice was to
>> unconditionally enable FOO support in host-python, adding build time to
>> everyone, even if FOO in host-python might only be needed for one
>> obscure package.
> 
> In this case, maybe we could simply depart from the rule, and just have
> python/Config.in.host contain just:
> 
>    # Select this if you need host-python to support 'stuff'
>    config BR2_NEEDS_HOST_PYTHON_WITH_STUFF
>        bool
> 
> And then, python/python.mk would have:
> 
>    ifeq ($(BR2_NEEDS_HOST_PYTHON_WITH_STUFF),y)
>    HOST_PYTHON_CONF_OPTS += --with-stuff
>    else
>    HOST_PYTHON_CONF_OPTS += --without-stuff
>    endif
> 
> Regards,
> Yann E. MORIN.
> 
>> It seems like we don't have a very conclusive decision on this topic at
>> this point.

Well, I'm happy that my work helped you guys to analyze the pros and
cons of having Config.in.host for all host packages. Looks like it's
not a good idea, so I will mark the patches as refused in patchwork.

-- 
Carlos Santos (Casantos) - DATACOM, P&D

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Thomas Petazzoni]

Hello Carlos,

On Thu, 3 Jan 2019 21:34:03 -0200 (BRST), Carlos Santos wrote:

> Well, I'm happy that my work helped you guys to analyze the pros and
> cons of having Config.in.host for all host packages. Looks like it's
> not a good idea, so I will mark the patches as refused in patchwork.

I don't think we can conclude yet "it's not a good idea". At this
point, my feeling is more: the consequences are worse than I initially
expected, which requires some more thoughts on how we want to handle
that.

Again, what was your motivation for adding those Config.in.host options
in the first place, specifically for waf ? Was it just about getting
this whole discussion started, and use waf just as a simple first
example ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Carlos Santos]

> From: "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
> To: "Carlos Santos" <casantos@datacom.com.br>
> Cc: "Yann Morin" <yann.morin.1998@free.fr>, "Peter Korsgaard" <peter@korsgaard.com>, "Mahyar Koshkouei"
> <mahyar.koshkouei@gmail.com>, "Thomas De Schampheleire" <thomas.de_schampheleire@nokia.com>, "buildroot"
> <buildroot@buildroot.org>, "Arnout Vandecappelle" <arnout@mind.be>
> Sent: Sexta-feira, 4 de janeiro de 2019 6:58:11
> Subject: Re: [Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

> Hello Carlos,
> 
> On Thu, 3 Jan 2019 21:34:03 -0200 (BRST), Carlos Santos wrote:
> 
>> Well, I'm happy that my work helped you guys to analyze the pros and
>> cons of having Config.in.host for all host packages. Looks like it's
>> not a good idea, so I will mark the patches as refused in patchwork.
> 
> I don't think we can conclude yet "it's not a good idea". At this
> point, my feeling is more: the consequences are worse than I initially
> expected, which requires some more thoughts on how we want to handle
> that.
> 
> Again, what was your motivation for adding those Config.in.host options
> in the first place, specifically for waf ? Was it just about getting
> this whole discussion started, and use waf just as a simple first
> example ?

It started when I realized that the trailing slashes in WAF_SITE and
other host packages. Then Arnout said "we need Config.in.host" and I
decided to give it a try. It may be a silly explanation but sometimes
we do things just for fun. :-)

-- 
Carlos Santos (Casantos) - DATACOM, P&D

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Thomas Petazzoni]

Hello,

On Fri, 4 Jan 2019 08:05:41 -0200 (BRST), Carlos Santos wrote:

> It started when I realized that the trailing slashes in WAF_SITE and
> other host packages. Then Arnout said "we need Config.in.host" and I
> decided to give it a try. It may be a silly explanation but sometimes
> we do things just for fun. :-)

No, no, that's a perfectly fine explanation :-)

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Arnout Vandecappelle]

On 03/01/2019 23:06, Yann E. MORIN wrote:
> Thomas, All,
> 
> On 2019-01-03 22:50 +0100, Thomas Petazzoni spake thusly:
>> On Wed, 26 Dec 2018 22:55:21 +0100, Yann E. MORIN wrote:
>>> Well, I am not really happy with that, though: do we really plan on
>>> having packages really select all the host tools they need?
>>>
>>> If so, do we really envision autotools-based packages selecting
>>> host-autoconf, host-automake, host-libtool? And then packages that use
>>> host-pkgconf you should also select it...
>>>
>>> Also, what about host-cmake, which is conditionally built, but for which
>>> we do not have the info in kconfig? (well, we can argue we'd have to do
>>> like we do for host-gcc, but still). Oh, and host-tar, host-flex,
>>> host-bison, and so on... :-/
>>
>> Meh, I hadn't thought of conditional packages like host-tar, host-cmake
>> and so on.

 I also hadn't thought of that. But it's not an unsurmountable problem.

>>
>>> So, no, I'm not happy with that direction...
>>>
>>>     config BR2_PACKAGE_FOO
>>>         bool "foo"
>>>         select BR2_PACKAGE_HOST_AUTOCONF
>>>         select BR2_PACKAGE_HOST_AUTOMAKE
>>>         select BR2_PACKAGE_HOST_LIBTOOL
>>>         select BR2_PACKAGE_MAYBE_HOST_TAR
>>>         select BR2_PACKAGE_MAYBE_HOST_FLEX_FOR_KCONFIG
>>>         select BR2_PACKAGE_MAYBE_HOST_BISON_FOR_KCONFIG
>>>         select BR2_PACKAGE_HOST_PKGCONF
>>>
>>> Unles we're planning on hiding that away into meta-config, like:
>>>
>>>     config BR2_PACKAGE_FOO
>>>         bool "foo"
>>>         select BR2_AUTOTOOLS_PACKAGE # mimick $(eval $(autotools-package))
>>>         select BR2_PACKAGE_HOST_PKGCONF_BECAUSE_IT_S_NOT_MANDATORY

 This looks acceptable to me...


>>> And still, the optionally-required host packages like tar, flex et al.
>>> are not covered...

 tar at least is rather easy to overcome: we can add environment variables that
pass the results of dependencies.mk to Kconfig. And for things that are
conditional like lzip we can use a combination of the two.

>>>
>>> Meh... :-(
>>
>> Indeed, I understand the "Meh" here. I hadn't really realized what it
>> would mean to have Config.in.host options for all packages, and
>> properly selected by all its users.
> 
> To be fait, I was initially also in favour of adding Config.in.host
> options for all host packages. But this waf patch made me change my
> mind, becasue of the above...

 Note, however, that the only reason to require Config.in.host for *all*
packages is that it allows us to have a (partial) check that for each make
dependency, the Kconfig dependency exists as well (i.e., check that all
dependencies are in PACKAGES). That's not a very strong reason. So I would be
perfectly fine with making this a "soft" requirement (possibly verified with
checkpackage).

 The tricky thing is that when a package does have a Config.in.host option, we
really want it to be selected in *all* cases that it is built. So ideally we
should have a way to verify that...


 Regards,
 Arnout

> 
>> But still, there are a number of cases where it would really help, so
>> that a given host package can be aware that another host package has
>> been built with a given feature (or not). Or precisely to force that a
>> certain host package is built with a given option. For example, in
>> host-python, we had situations where only a given package needed
>> host-python to be built with FOO support, and since we don't have any
>> BR2_PACKAGE_HOST_PYTHON_FOO option, our only choice was to
>> unconditionally enable FOO support in host-python, adding build time to
>> everyone, even if FOO in host-python might only be needed for one
>> obscure package.
> 
> In this case, maybe we could simply depart from the rule, and just have
> python/Config.in.host contain just:
> 
>     # Select this if you need host-python to support 'stuff'
>     config BR2_NEEDS_HOST_PYTHON_WITH_STUFF
>         bool
> 
> And then, python/python.mk would have:
> 
>     ifeq ($(BR2_NEEDS_HOST_PYTHON_WITH_STUFF),y)
>     HOST_PYTHON_CONF_OPTS += --with-stuff
>     else
>     HOST_PYTHON_CONF_OPTS += --without-stuff
>     endif
> 
> Regards,
> Yann E. MORIN.
> 
>> It seems like we don't have a very conclusive decision on this topic at
>> this point.
> 
> 

[Buildroot] [PATCH v3 1/3] package/waf: add a blind Config.in.host

[Yann E. MORIN]

Arnout, All,

On 2019-01-04 12:06 +0100, Arnout Vandecappelle spake thusly:
> On 03/01/2019 23:06, Yann E. MORIN wrote:
> > On 2019-01-03 22:50 +0100, Thomas Petazzoni spake thusly:
> >> On Wed, 26 Dec 2018 22:55:21 +0100, Yann E. MORIN wrote:
[--SNIP--]
> >>> So, no, I'm not happy with that direction...
> >>>
> >>>     config BR2_PACKAGE_FOO
> >>>         bool "foo"
> >>>         select BR2_PACKAGE_HOST_AUTOCONF
> >>>         select BR2_PACKAGE_HOST_AUTOMAKE
> >>>         select BR2_PACKAGE_HOST_LIBTOOL
> >>>         select BR2_PACKAGE_MAYBE_HOST_TAR
> >>>         select BR2_PACKAGE_MAYBE_HOST_FLEX_FOR_KCONFIG
> >>>         select BR2_PACKAGE_MAYBE_HOST_BISON_FOR_KCONFIG
> >>>         select BR2_PACKAGE_HOST_PKGCONF
> >>>
> >>> Unles we're planning on hiding that away into meta-config, like:
> >>>
> >>>     config BR2_PACKAGE_FOO
> >>>         bool "foo"
> >>>         select BR2_AUTOTOOLS_PACKAGE # mimick $(eval $(autotools-package))
> >>>         select BR2_PACKAGE_HOST_PKGCONF_BECAUSE_IT_S_NOT_MANDATORY
> 
>  This looks acceptable to me...

Except this is still missing a lot of things (I just got bored writing
the stuff).

> >>> And still, the optionally-required host packages like tar, flex et al.
> >>> are not covered...
> 
>  tar at least is rather easy to overcome: we can add environment variables that
> pass the results of dependencies.mk to Kconfig. And for things that are
> conditional like lzip we can use a combination of the two.

Well, the problem is not how to pass the information to Kconfig: we
already know how to do it (e.g. host gcc version). The problem is that
packages should select the host-packages they need, then:

  - it is very verbose in packages, as you can see above,

  - we need to invent a new semantic for conditional packages.

[--SNIP--]
>  Note, however, that the only reason to require Config.in.host for *all*
> packages is that it allows us to have a (partial) check that for each make

Why do you say "partial" in "a (partial) check"? If we can't do aproper
and complete check, then there is no reason to add Config.in.host for
all packages...

> dependency, the Kconfig dependency exists as well (i.e., check that all
> dependencies are in PACKAGES).

And then, what does that brings us?

> That's not a very strong reason. So I would be
> perfectly fine with making this a "soft" requirement (possibly verified with
> checkpackage).
> 
>  The tricky thing is that when a package does have a Config.in.host option, we
> really want it to be selected in *all* cases that it is built. So ideally we
> should have a way to verify that...

Yes, the same as we currently check that all packages in FOO_DEPENDENCIES
are indeed enabled when they do ahve a corresonding *_KCONFIG_VAR, see:

    https://git.buildroot.org/buildroot/tree/Makefile#n547

Except the check is now limted to target packages only.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'