[Buildroot] [PATCH 0/3] Speed-up gettext and host-gettext build

[Buildroot] [PATCH 0/3] Speed-up gettext and host-gettext build

[Thomas Petazzoni]

Hello,

This series aims at speeding up the build of the gettext and
host-gettext packages, which are particularly long currently.

It has been tested with the minidlna build against a uClibc toolchain,
minidlna being a package that needs both gettext and host-gettext.

It would be nice if a few other people could test these changes in
their own configuration, to confirm that they don't break anything.

Best regards,

Thomas

Thomas Petazzoni (3):
  minidlna: do not select BR2_PACKAGE_GETTEXT_TOOLS
  gettext: remove support for gettext-tools on target
  gettext: optimize build time

 Config.in.legacy                                   | 12 +++++++
 package/gettext/Config.in                          | 14 --------
 .../gettext-03-option-to-disable-tools.patch       | 42 ----------------------
 .../gettext/gettext-04-remove-examples-tests.patch | 20 -----------
 package/gettext/gettext.mk                         | 27 +++++++-------
 package/minidlna/Config.in                         |  1 -
 6 files changed, 27 insertions(+), 89 deletions(-)
 delete mode 100644 package/gettext/gettext-03-option-to-disable-tools.patch
 delete mode 100644 package/gettext/gettext-04-remove-examples-tests.patch

-- 
1.9.3

[Buildroot] [PATCH 1/3] minidlna: do not select BR2_PACKAGE_GETTEXT_TOOLS

[Thomas Petazzoni]

Commit 9ff640ae30ca588228030faf2c6fd13af292cb66 ("minidlna: needs
gettext and host-gettext tools if locale is enabled") modified the
minidlna package to fix gettext related issues. As part of this patch,
a select of BR2_PACKAGE_GETTEXT_TOOLS was added, which according to
the commit title is used to have host-gettext tools installed.

However, this is not what this option is about: this option is about
having gettext tools installed on the target.

Since this is not what minidlna needs, and we anyway plan to remove
this BR2_PACKAGE_GETTEXT_TOOLS option, this commit removes this
incorrect select statement.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Bernd Kuhls <berndkuhls@hotmail.com>
---
 package/minidlna/Config.in | 1 -
 1 file changed, 1 deletion(-)

diff --git a/package/minidlna/Config.in b/package/minidlna/Config.in
index 4a46b0c..5c5306d 100644
--- a/package/minidlna/Config.in
+++ b/package/minidlna/Config.in
@@ -14,7 +14,6 @@ config BR2_PACKAGE_MINIDLNA
 	select BR2_PACKAGE_LIBJPEG
 	select BR2_PACKAGE_SQLITE
 	select BR2_PACKAGE_GETTEXT if BR2_NEEDS_GETTEXT_IF_LOCALE
-	select BR2_PACKAGE_GETTEXT_TOOLS if BR2_NEEDS_GETTEXT_IF_LOCALE
 	help
 	  MiniDLNA (aka ReadyDLNA) is server software with the aim of being
 	  fully compliant with DLNA/UPnP-AV clients.
-- 
1.9.3

[Buildroot] [PATCH 2/3] gettext: remove support for gettext-tools on target

[Thomas Petazzoni]

This commit removes the BR2_PACKAGE_GETTEXT_TOOLS option, which could
be used to install gettext tools on the target. This is not needed,
because Buildroot is not designed to provide a full development
environment on the target, and gettext translation files should be
processed on the build machine, using the host gettext tools.

Remove this option will allow to optimize the build time of gettext on
the target, by only building the gettext runtime libraries.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Config.in.legacy           | 12 ++++++++++++
 package/gettext/Config.in  | 14 --------------
 package/gettext/gettext.mk |  4 ----
 3 files changed, 12 insertions(+), 18 deletions(-)

diff --git a/Config.in.legacy b/Config.in.legacy
index 00661db..10e495b 100644
--- a/Config.in.legacy
+++ b/Config.in.legacy
@@ -99,6 +99,18 @@ comment "----------------------------------------------------"
 endif
 
 ###############################################################################
+comment "Legacy options removed in 2014.08"
+
+config BR2_PACKAGE_GETTEXT_TOOLS
+	bool "gettext-tools on target support removed"
+	select BR2_LEGACY
+	help
+	  The option to install the gettext utilities on the target
+	  has been removed. This is not necessary as Buildroot is not
+	  designed to provide a full development environment on the
+	  target. gettext tools should be used on the build machine
+	  instead.
+
 comment "Legacy options removed in 2014.05"
 
 config BR2_PACKAGE_EVTEST_CAPTURE
diff --git a/package/gettext/Config.in b/package/gettext/Config.in
index d33dd3a..158b14e 100644
--- a/package/gettext/Config.in
+++ b/package/gettext/Config.in
@@ -13,20 +13,6 @@ config BR2_PACKAGE_GETTEXT
 
 	  http://www.gnu.org/software/gettext/
 
-if BR2_PACKAGE_GETTEXT
-
-config BR2_PACKAGE_GETTEXT_TOOLS
-	bool "Install gettext tools"
-	depends on BR2_USE_MMU # fork()
-	depends on BR2_LARGEFILE
-	select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE
-	help
-	  This option allows to install the complete gettext suite in
-	  the target filesystem. This is typically not useful for
-	  correct operation of programs.
-
-endif
-
 comment "gettext needs a toolchain w/ wchar"
 	depends on BR2_NEEDS_GETTEXT
 	depends on !BR2_USE_WCHAR
diff --git a/package/gettext/gettext.mk b/package/gettext/gettext.mk
index b5efa76..52a72db 100644
--- a/package/gettext/gettext.mk
+++ b/package/gettext/gettext.mk
@@ -46,16 +46,12 @@ ifeq ($(BR2_ENABLE_LOCALE),)
 GETTEXT_CONF_OPT += --enable-nls
 endif
 
-# When the gettext tools are not enabled in the configuration, we only
-# install libintl to the target.
-ifeq ($(BR2_PACKAGE_GETTEXT_TOOLS),)
 # When static libs are preferred the .so files aren't created
 ifeq ($(BR2_PREFER_STATIC_LIB),)
 define GETTEXT_INSTALL_TARGET_CMDS
 	cp -dpf $(STAGING_DIR)/usr/lib/libintl*.so* $(TARGET_DIR)/usr/lib/
 endef
 endif
-endif # GETTEXT_TOOLS = n
 
 # Disable interactive confirmation in host gettextize for package fixups
 define HOST_GETTEXT_GETTEXTIZE_CONFIRMATION
-- 
1.9.3

[Buildroot] [PATCH 3/3] gettext: optimize build time

[Thomas Petazzoni]

This commit significantly reduces the build time of host-gettext and
gettext, by using the capacity of gettext to handle build things in a
certain subdirectory:

 - For the host variant of gettext, we only need the gettext-tools,
   available in the directory of the same name in the gettext sources.

 - For the target variant of gettext, we only need the gettext library
   libintl, available in the gettext-runtime directory in the gettext
   sources.

So by using appropriate values of GETTEXT_SUBDIR and
HOST_GETTEXT_SUBDIR, we only build what's necessary. Moreover, by
manually patching gettext-tools/Makefile.in and
gettext-runtime/Makefile.in, we make sure to not build and install
things like examples, documentation and so on.

In addition to this, these changes avoid the need to autoreconfigure
the gettext package, which was particularly long.

Thanks to these changes, the build time of gettext goes from 1 minutes
and 37 seconds to just 24 seconds, and the build of host-gettext goes
from 2 minutes and 18 seconds to 1 minute and 13 seconds.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../gettext-03-option-to-disable-tools.patch       | 42 ----------------------
 .../gettext/gettext-04-remove-examples-tests.patch | 20 -----------
 package/gettext/gettext.mk                         | 23 +++++++-----
 3 files changed, 15 insertions(+), 70 deletions(-)
 delete mode 100644 package/gettext/gettext-03-option-to-disable-tools.patch
 delete mode 100644 package/gettext/gettext-04-remove-examples-tests.patch

diff --git a/package/gettext/gettext-03-option-to-disable-tools.patch b/package/gettext/gettext-03-option-to-disable-tools.patch
deleted file mode 100644
index 24a9996..0000000
--- a/package/gettext/gettext-03-option-to-disable-tools.patch
+++ /dev/null
@@ -1,42 +0,0 @@
-Add a --disable-tools option
-
-This patch adds a --disable-tools option that allows to disable the
-compilation of the gettext tools, which may not be necessary on a
-small embedded Linux system.
-
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Index: b/configure.ac
-===================================================================
---- a/configure.ac
-+++ b/configure.ac
-@@ -37,6 +37,10 @@
- 
- dnl Checks for library functions.
- 
-+AC_ARG_ENABLE([tools], [--disable-tools  do not build tools],
-+		       [enable_tools=$enableval], [enable_tools=yes])
-+AM_CONDITIONAL([ENABLE_TOOLS], [test "$enable_tools" = "yes"])
-+
- AC_CONFIG_SUBDIRS([gettext-runtime gettext-tools])
- 
- dnl Ensure that "configure --help" lists all the command line options that
-Index: b/Makefile.am
-===================================================================
---- a/Makefile.am
-+++ b/Makefile.am
-@@ -20,7 +20,13 @@
- ACLOCAL = build-aux/fixaclocal @ACLOCAL@
- ACLOCAL_AMFLAGS = -I m4
- 
--SUBDIRS = gnulib-local gettext-runtime gettext-tools
-+if ENABLE_TOOLS
-+TOOLS_SUBDIR = gettext-tools
-+else
-+TOOLS_SUBDIR =
-+endif
-+
-+SUBDIRS = gnulib-local gettext-runtime $(TOOLS_SUBDIR)
- 
- EXTRA_DIST = \
-   version.sh DEPENDENCIES PACKAGING HACKING ChangeLog.0 autogen.sh \
diff --git a/package/gettext/gettext-04-remove-examples-tests.patch b/package/gettext/gettext-04-remove-examples-tests.patch
deleted file mode 100644
index f7d88d9..0000000
--- a/package/gettext/gettext-04-remove-examples-tests.patch
+++ /dev/null
@@ -1,20 +0,0 @@
-Disable tests and examples
-
-This patch disables the gettext-tools tests and examples, which are
-generally not necessary.
-
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Index: b/gettext-tools/Makefile.am
-===================================================================
---- a/gettext-tools/Makefile.am
-+++ b/gettext-tools/Makefile.am
-@@ -20,7 +20,7 @@
- ACLOCAL = ../build-aux/fixaclocal @ACLOCAL@
- ACLOCAL_AMFLAGS = -I m4 -I ../gettext-runtime/m4 -I ../m4 -I gnulib-m4 -I libgrep/gnulib-m4 -I libgettextpo/gnulib-m4
- 
--SUBDIRS = doc intl gnulib-lib libgrep src libgettextpo po projects styles misc man m4 tests gnulib-tests examples
-+SUBDIRS = doc intl gnulib-lib libgrep src libgettextpo po projects styles misc man m4
- 
- EXTRA_DIST = misc/DISCLAIM
- MOSTLYCLEANFILES = core *.stackdump
diff --git a/package/gettext/gettext.mk b/package/gettext/gettext.mk
index 52a72db..bbcbdd9 100644
--- a/package/gettext/gettext.mk
+++ b/package/gettext/gettext.mk
@@ -9,7 +9,6 @@ GETTEXT_SITE = $(BR2_GNU_MIRROR)/gettext
 GETTEXT_INSTALL_STAGING = YES
 GETTEXT_LICENSE = GPLv2+
 GETTEXT_LICENSE_FILES = COPYING
-GETTEXT_AUTORECONF = YES
 
 GETTEXT_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv)
 HOST_GETTEXT_DEPENDENCIES = # we don't want the libiconv dependency
@@ -37,6 +36,21 @@ HOST_GETTEXT_CONF_OPT = \
 	--disable-relocatable \
 	--without-emacs
 
+# For the target version, we only need the runtime, and for the host
+# version, we only need the tools.
+GETTEXT_SUBDIR = gettext-runtime
+HOST_GETTEXT_SUBDIR = gettext-tools
+
+# Disable the build of documentation and examples of gettext-tools,
+# and the build of documentation and tests of gettext-runtime.
+define HOST_GETTEXT_DISABLE_UNNEEDED
+	$(SED) '/^SUBDIRS/s/ doc //;/^SUBDIRS/s/examples$$//' $(@D)/gettext-tools/Makefile.in
+	$(SED) '/^SUBDIRS/s/ doc //;/^SUBDIRS/s/tests$$//' $(@D)/gettext-runtime/Makefile.in
+endef
+
+GETTEXT_POST_PATCH_HOOKS += HOST_GETTEXT_DISABLE_UNNEEDED
+HOST_GETTEXT_POST_PATCH_HOOKS += HOST_GETTEXT_DISABLE_UNNEEDED
+
 # Force build with NLS support, otherwise libintl is not built
 # This is needed because some packages (eg. libglib2) requires
 # locales, but do not properly depend on BR2_ENABLE_LOCALE, and
@@ -46,13 +60,6 @@ ifeq ($(BR2_ENABLE_LOCALE),)
 GETTEXT_CONF_OPT += --enable-nls
 endif
 
-# When static libs are preferred the .so files aren't created
-ifeq ($(BR2_PREFER_STATIC_LIB),)
-define GETTEXT_INSTALL_TARGET_CMDS
-	cp -dpf $(STAGING_DIR)/usr/lib/libintl*.so* $(TARGET_DIR)/usr/lib/
-endef
-endif
-
 # Disable interactive confirmation in host gettextize for package fixups
 define HOST_GETTEXT_GETTEXTIZE_CONFIRMATION
 	$(SED) '/read dummy/d' $(HOST_DIR)/usr/bin/gettextize
-- 
1.9.3

[Buildroot] [PATCH 3/3] gettext: optimize build time

[Baruch Siach]

Hi Thomas,

On Thu, May 29, 2014 at 12:40:28PM +0200, Thomas Petazzoni wrote:
> This commit significantly reduces the build time of host-gettext and
> gettext, by using the capacity of gettext to handle build things in a
> certain subdirectory:
> 
>  - For the host variant of gettext, we only need the gettext-tools,
>    available in the directory of the same name in the gettext sources.
> 
>  - For the target variant of gettext, we only need the gettext library
>    libintl, available in the gettext-runtime directory in the gettext
>    sources.
> 
> So by using appropriate values of GETTEXT_SUBDIR and
> HOST_GETTEXT_SUBDIR, we only build what's necessary. Moreover, by
> manually patching gettext-tools/Makefile.in and
> gettext-runtime/Makefile.in, we make sure to not build and install
> things like examples, documentation and so on.
> 
> In addition to this, these changes avoid the need to autoreconfigure
> the gettext package, which was particularly long.
> 
> Thanks to these changes, the build time of gettext goes from 1 minutes
> and 37 seconds to just 24 seconds, and the build of host-gettext goes
> from 2 minutes and 18 seconds to 1 minute and 13 seconds.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  .../gettext-03-option-to-disable-tools.patch       | 42 ----------------------

Why not remove --disable-tools from GETTEXT_CONF_OPT then?

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

[Buildroot] [PATCH 3/3] gettext: optimize build time

[Thomas Petazzoni]

Dear Baruch Siach,

On Thu, 29 May 2014 13:56:23 +0300, Baruch Siach wrote:

> > Thanks to these changes, the build time of gettext goes from 1 minutes
> > and 37 seconds to just 24 seconds, and the build of host-gettext goes
> > from 2 minutes and 18 seconds to 1 minute and 13 seconds.
> > 
> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > ---
> >  .../gettext-03-option-to-disable-tools.patch       | 42 ----------------------
> 
> Why not remove --disable-tools from GETTEXT_CONF_OPT then?

Indeed, thanks for noticing. I'll fix that up in v2.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/3] gettext: remove support for gettext-tools on target

[Thomas De Schampheleire]

On Thu, May 29, 2014 at 12:40 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> This commit removes the BR2_PACKAGE_GETTEXT_TOOLS option, which could
> be used to install gettext tools on the target. This is not needed,
> because Buildroot is not designed to provide a full development
> environment on the target, and gettext translation files should be
> processed on the build machine, using the host gettext tools.
>
> Remove this option will allow to optimize the build time of gettext on
> the target, by only building the gettext runtime libraries.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  Config.in.legacy           | 12 ++++++++++++
>  package/gettext/Config.in  | 14 --------------
>  package/gettext/gettext.mk |  4 ----
>  3 files changed, 12 insertions(+), 18 deletions(-)
>
> diff --git a/Config.in.legacy b/Config.in.legacy
> index 00661db..10e495b 100644
> --- a/Config.in.legacy
> +++ b/Config.in.legacy
> @@ -99,6 +99,18 @@ comment "----------------------------------------------------"
>  endif
>
>  ###############################################################################
> +comment "Legacy options removed in 2014.08"
> +
> +config BR2_PACKAGE_GETTEXT_TOOLS
> +       bool "gettext-tools on target support removed"

I would rewrite this as any of the following:

support for gettext-tools on target has been removed
gettext-tools target support has been removed
gettext-tools target option has been removed

Best regards,
Thomas

[Buildroot] [PATCH 0/3] Speed-up gettext and host-gettext build

[Andrew Ruder]

On Thu, May 29, 2014 at 12:40:25PM +0200, Thomas Petazzoni wrote:
> It would be nice if a few other people could test these changes in
> their own configuration, to confirm that they don't break anything.

Tested-by: Andrew Ruder <andrew.ruder@elecsyscorp.com>

Cuts my build of host-gettext from ~115 seconds to ~55 seconds.

Thanks,
Andy