[Buildroot] [PATCH 0/3] [RFC] pkg-autotools: introduce infra to gettextize packages (branch yem/gettextize)

[Buildroot] [PATCH 0/3] [RFC] pkg-autotools: introduce infra to gettextize packages (branch yem/gettextize)

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Hello All!

This small series introduces the gettextize of a package as an optional
sequence in the autotools package infra.

    WARNING! Completely untested, only an RFC!

Currently, wget is the sole user, and the way it is done for wget is
just working by chance (see commit log of path 3 for the complete
explanations.)

So, move the gettextization to the pkg-autotools infra, so we have a
chance to do it correctly and consistently across packages.

Regards,
Yann E. MORIN.


The following changes since commit 8b083158a5d7c80bc54697c17ab4bbbc8d65fb04:

  linux-headers: bump 3.{2,12}.x series (2014-04-09 13:45:13 +0200)

are available in the git repository at:

  git://gitorious.org/buildroot/buildroot.git yem/gettextize

for you to fetch changes up to f318a05b46d762c07c435c133b0a6e836600c6d6:

  package/wget: use the new gettextize infra (2014-04-10 00:16:01 +0200)

----------------------------------------------------------------
Yann E. MORIN (3):
      pkg-autotools: add support to gettextize a package
      manual: add gettextize explanations in the manual
      package/wget: use the new gettextize infra

 docs/manual/adding-packages-autotools.txt | 12 ++++++++++++
 package/pkg-autotools.mk                  | 30 ++++++++++++++++++++++++++++++
 package/wget/wget.mk                      |  6 +-----
 3 files changed, 43 insertions(+), 5 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/3] pkg-autotools: add support to gettextize a package

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

In case a package comes with a gettext infra which is different from
the one Buildroot provides, we'd get autoreconf errors, like:
    http://autobuild.buildroot.net/results/c0f/c0f7c801f61fdc310cde64342060b00a70155431/

To avoid that, we need to gettextize the package prior to running
autoreconf.

Provide the necessary infrastructure in the autotools infrastructure,
so we can do it consistently across packages.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Baruch Siach <baruch@tkos.co.il>
---
 package/pkg-autotools.mk | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/package/pkg-autotools.mk b/package/pkg-autotools.mk
index a646612..84c5577 100644
--- a/package/pkg-autotools.mk
+++ b/package/pkg-autotools.mk
@@ -87,6 +87,22 @@ ifndef $(2)_AUTORECONF
  endif
 endif
 
+ifndef $(2)_GETTEXTIZE
+ ifdef $(3)_GETTEXTIZE
+  $(2)_GETTEXTIZE = $($(3)_GETTEXTIZE)
+ else
+  $(2)_GETTEXTIZE ?= NO
+ endif
+endif
+
+ifndef $(2)_GETTEXTIZE_OPT
+ ifdef $(3)_GETTEXTIZE_OPT
+  $(2)_GETTEXTIZE_OPT = $($(3)_GETTEXTIZE)
+ else
+  $(2)_GETTEXTIZE_OPT ?= -f
+ endif
+endif
+
 $(2)_CONF_ENV			?=
 $(2)_CONF_OPT			?=
 $(2)_MAKE_ENV			?=
@@ -197,6 +213,20 @@ $(2)_POST_PATCH_HOOKS += LIBTOOL_PATCH_HOOK
 endif
 
 #
+# Hook to gettextize the package if needed
+#
+define GETTEXTIZE_HOOK
+	@$$(call MESSAGE,"Gettextizing")
+	$(Q)cd $$($$(PKG)_SRCDIR) && $(HOST_DIR)/usr/bin/gettextize $$($$(PKG)_GETTEXTIZE_OPT)
+endef
+
+# This has to come before autoreconf
+ifeq ($$($(2)_GETTEXTIZE),YES)
+$(2)_PRE_CONFIGURE_HOOK += GETTEXTIZE_HOOK
+$(2)_DEPENDENCIES += host-gettext
+endif
+
+#
 # Hook to autoreconf the package if needed
 #
 define AUTORECONF_HOOK
-- 
1.8.3.2

[Buildroot] [PATCH 2/3] manual: add gettextize explanations in the manual

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Baruch Siach <baruch@tkos.co.il>
---
 docs/manual/adding-packages-autotools.txt | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/docs/manual/adding-packages-autotools.txt b/docs/manual/adding-packages-autotools.txt
index cc668b5..9181fb8 100644
--- a/docs/manual/adding-packages-autotools.txt
+++ b/docs/manual/adding-packages-autotools.txt
@@ -128,6 +128,18 @@ cases, typical packages will therefore only use a few of them.
   passed to the 'autoreconf' program if
   +LIBFOO_AUTORECONF=YES+. By default, empty.
 
+* +LIBFOO_GETTEXTIZE+, tells whether the package should be
+  gettextized or not (i.e. if the pacakge uses an different gettext
+  version than Buildroot provides, and it is needed to run
+  'gettextize'.) Valid values are +YES+ and   +NO+. The default
+  is +NO+.
+
+* +LIBFOO_GETTEXTIZE_OPT+, to specify additional options passed to
+  the 'gettextize' program, if +LIBFOO_GETTEXTIZE=YES+. You may
+  use that if, for example, the +.po+ files are not located in the
+  standard place (i.e. in +po/+ at the root of the package.) By
+  default, '-f'.
+
 * +LIBFOO_LIBTOOL_PATCH+ tells whether the Buildroot
   patch to fix libtool cross-compilation issues should be applied or
   not. Valid values are +YES+ and +NO+. By
-- 
1.8.3.2

[Buildroot] [PATCH 2/3] manual: add gettextize explanations in the manual

[Thomas De Schampheleire]

Hi Yann,

On Thu, Apr 10, 2014 at 12:20 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Baruch Siach <baruch@tkos.co.il>
> ---
>  docs/manual/adding-packages-autotools.txt | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/docs/manual/adding-packages-autotools.txt b/docs/manual/adding-packages-autotools.txt
> index cc668b5..9181fb8 100644
> --- a/docs/manual/adding-packages-autotools.txt
> +++ b/docs/manual/adding-packages-autotools.txt
> @@ -128,6 +128,18 @@ cases, typical packages will therefore only use a few of them.
>    passed to the 'autoreconf' program if
>    +LIBFOO_AUTORECONF=YES+. By default, empty.
>
> +* +LIBFOO_GETTEXTIZE+, tells whether the package should be
> +  gettextized or not (i.e. if the pacakge uses an different gettext

package
a different

> +  version than Buildroot provides, and it is needed to run
> +  'gettextize'.) Valid values are +YES+ and   +NO+. The default

There seem to be extra spaces between 'and' and +NO+.

> +  is +NO+.
> +
> +* +LIBFOO_GETTEXTIZE_OPT+, to specify additional options passed to
> +  the 'gettextize' program, if +LIBFOO_GETTEXTIZE=YES+. You may
> +  use that if, for example, the +.po+ files are not located in the
> +  standard place (i.e. in +po/+ at the root of the package.) By
> +  default, '-f'.
> +
>  * +LIBFOO_LIBTOOL_PATCH+ tells whether the Buildroot
>    patch to fix libtool cross-compilation issues should be applied or
>    not. Valid values are +YES+ and +NO+. By


Best regards,
Thomas

[Buildroot] [PATCH 3/3] package/wget: use the new gettextize infra

[Yann E. MORIN]

From: "Yann E. MORIN" <yann.morin.1998@free.fr>

Currently, the gettextization of wget works by chance:
  - host-gettext is added as a dependency to wget;
  - gettextize is run as a post-patch hook.

But the dependencies are only guaranteed to be built and installed
for the configure step, not the patch step. Becasue post-patch hooks
are part of the patch step, we have no guarantee that the dependency
to host-gettext is done by the time we gettextize wget.

This happens to work by chance, since wget sorts alphabetically after
gettext, so we indeed have host-gettext built and installed by the
time we need to gettextize wget.

This is prone to fail in the parallel build case, sicne we can no
longer rely on alphabetical order in that case.

Instead, use the new gettextize infra we just added.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/wget/wget.mk | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/package/wget/wget.mk b/package/wget/wget.mk
index f3a5274..031a8ea 100644
--- a/package/wget/wget.mk
+++ b/package/wget/wget.mk
@@ -15,11 +15,7 @@ WGET_AUTORECONF = YES
 
 # Ugly kludge to fix autoreconf with old gettext infra
 # We need to gettextize before autoreconf to upgrade
-WGET_DEPENDENCIES += host-gettext
-define WGET_GETTEXTIZE
-	cd $(@D) ; $(HOST_DIR)/usr/bin/gettextize -f
-endef
-WGET_POST_PATCH_HOOKS += WGET_GETTEXTIZE
+WGET_GETTEXTIZE = YES
 
 # Prefer full-blown wget over busybox
 ifeq ($(BR2_PACKAGE_BUSYBOX),y)
-- 
1.8.3.2

[Buildroot] [PATCH 3/3] package/wget: use the new gettextize infra

[Baruch Siach]

Hi Yann,

Thanks for noticing.

On Thu, Apr 10, 2014 at 12:20:05AM +0200, Yann E. MORIN wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> 
> Currently, the gettextization of wget works by chance:
>   - host-gettext is added as a dependency to wget;
>   - gettextize is run as a post-patch hook.
> 
> But the dependencies are only guaranteed to be built and installed
> for the configure step, not the patch step. Becasue post-patch hooks
> are part of the patch step, we have no guarantee that the dependency
> to host-gettext is done by the time we gettextize wget.
> 
> This happens to work by chance, since wget sorts alphabetically after
> gettext, so we indeed have host-gettext built and installed by the
> time we need to gettextize wget.
> 
> This is prone to fail in the parallel build case, sicne we can no
> longer rely on alphabetical order in that case.

Fixing this only requires using PRE_CONFIGURE_HOOK instead of POST_PATCH_HOOK, 
isn't it? Is there anything else this series attempts to fix?

baruch

> Instead, use the new gettextize infra we just added.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> ---
>  package/wget/wget.mk | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/package/wget/wget.mk b/package/wget/wget.mk
> index f3a5274..031a8ea 100644
> --- a/package/wget/wget.mk
> +++ b/package/wget/wget.mk
> @@ -15,11 +15,7 @@ WGET_AUTORECONF = YES
>  
>  # Ugly kludge to fix autoreconf with old gettext infra
>  # We need to gettextize before autoreconf to upgrade
> -WGET_DEPENDENCIES += host-gettext
> -define WGET_GETTEXTIZE
> -	cd $(@D) ; $(HOST_DIR)/usr/bin/gettextize -f
> -endef
> -WGET_POST_PATCH_HOOKS += WGET_GETTEXTIZE
> +WGET_GETTEXTIZE = YES
>  
>  # Prefer full-blown wget over busybox
>  ifeq ($(BR2_PACKAGE_BUSYBOX),y)
> -- 
> 1.8.3.2

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

[Buildroot] [PATCH 3/3] package/wget: use the new gettextize infra

[Yann E. MORIN]

Baruch, All,

On 2014-04-10 06:34 +0300, Baruch Siach spake thusly:
> On Thu, Apr 10, 2014 at 12:20:05AM +0200, Yann E. MORIN wrote:
> > From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > 
> > Currently, the gettextization of wget works by chance:
> >   - host-gettext is added as a dependency to wget;
> >   - gettextize is run as a post-patch hook.
> > 
> > But the dependencies are only guaranteed to be built and installed
> > for the configure step, not the patch step. Becasue post-patch hooks
> > are part of the patch step, we have no guarantee that the dependency
> > to host-gettext is done by the time we gettextize wget.
> > 
> > This happens to work by chance, since wget sorts alphabetically after
> > gettext, so we indeed have host-gettext built and installed by the
> > time we need to gettextize wget.
> > 
> > This is prone to fail in the parallel build case, sicne we can no
> > longer rely on alphabetical order in that case.
> 
> Fixing this only requires using PRE_CONFIGURE_HOOK instead of POST_PATCH_HOOK, 
> isn't it? Is there anything else this series attempts to fix?

Yes, switching it to a pre-configure hook would fix it.
But that would be a one-off solution.

Providing it in the autotools-infra will allow us to have consistency
across packages and would allow us to fix it in a single place in the
future, rather than having to hunt down allow the offenders.

But as I said, this is an RFC, so we can discuss it further.

First, we have to agree we want it in the autotools infra or not.

Then, we'll have to check whether a boolean + an _OPT is enough, or if
we need something a bit more involved (e.g. let packages define the
complete hook, and just have the infra call it at the proper moment.)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/3] [RFC] pkg-autotools: introduce infra to gettextize packages (branch yem/gettextize)

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Thu, 10 Apr 2014 00:20:00 +0200, Yann E. MORIN wrote:

> This small series introduces the gettextize of a package as an optional
> sequence in the autotools package infra.
> 
>     WARNING! Completely untested, only an RFC!
> 
> Currently, wget is the sole user, and the way it is done for wget is
> just working by chance (see commit log of path 3 for the complete
> explanations.)
> 
> So, move the gettextization to the pkg-autotools infra, so we have a
> chance to do it correctly and consistently across packages.

I haven't looked at the details. But to me, when only one package needs
something, it is *way* too early to make changes to the common
infrastructure. Especially when there is a solution that allows to
solve the problem by making changes to the package .mk file itself.

I believe changes to the package infrastructure should only be made
when:

 * There is really no solution to solve the particular problem at the
   package .mk file level;

 * There are a sufficient (say 5 or 10 maybe) packages that have a
   similar problem, and therefore factorization would make sense.

So far, I don't really see a compelling reason to complexity the
package infrastructure just for the need of one particular package.
Need that may very well be transient, because a future bump of wget
will most likely remove the need for the gnulib patch, therefore for
the autoreconf.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com