[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Vadim Kochan]

From: Vadim Kochan <vadim.kochan@petcube.com>

Add gettext-tiny package from the sabotage-linux project:

gettext-tiny provides lightweight replacements for tools typically used
from the GNU gettext suite, which is incredibly bloated and takes a lot
of time to build (in the order of an hour on slow devices). the most
notable component is msgfmt which is used to create binary translation
files in the .mo format out of textual input files in .po format. this
is the most important tool for building software from source, because it
is used from the build processes of many software packages.

Signed-off-by: Vadim Kochan <vadim.kochan@petcube.com>
---
 DEVELOPERS                             |  3 +++
 package/Config.in                      |  1 +
 package/gettext-tiny/Config.in         | 10 +++++++++
 package/gettext-tiny/gettext-tiny.hash |  2 ++
 package/gettext-tiny/gettext-tiny.mk   | 38 ++++++++++++++++++++++++++++++++++
 5 files changed, 54 insertions(+)
 create mode 100644 package/gettext-tiny/Config.in
 create mode 100644 package/gettext-tiny/gettext-tiny.hash
 create mode 100644 package/gettext-tiny/gettext-tiny.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index 215506c71c..b84f8ebb7e 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -2010,6 +2010,9 @@ N:	Tzu-Jung Lee <roylee17@gmail.com>
 F:	package/dropwatch/
 F:	package/tstools/
 
+N:	Vadim Kochan <vadim4j@gmail.com>
+F:	package/gettext-tiny/
+
 N:	Valentin Korenblit <valentinkorenblit@gmail.com>
 F:	package/clang/
 F:	package/llvm/
diff --git a/package/Config.in b/package/Config.in
index f5a17492c7..9f29f93511 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -150,6 +150,7 @@ menu "Development tools"
 	source "package/flex/Config.in"
 	source "package/gawk/Config.in"
 	source "package/gettext/Config.in"
+	source "package/gettext-tiny/Config.in"
 	source "package/git/Config.in"
 	source "package/git-crypt/Config.in"
 	source "package/gperf/Config.in"
diff --git a/package/gettext-tiny/Config.in b/package/gettext-tiny/Config.in
new file mode 100644
index 0000000000..641403f1e6
--- /dev/null
+++ b/package/gettext-tiny/Config.in
@@ -0,0 +1,10 @@
+config BR2_PACKAGE_GETTEXT_TINY
+	bool "gettext-tiny"
+	select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE
+	help
+	  gettext-tiny provides lightweight replacements for tools typically
+	  used from the GNU gettext suite, which is incredibly bloated and
+	  takes a lot of time to build (in the order of an hour on slow
+	  devices).
+
+	  https://github.com/sabotage-linux/gettext-tiny
diff --git a/package/gettext-tiny/gettext-tiny.hash b/package/gettext-tiny/gettext-tiny.hash
new file mode 100644
index 0000000000..3b0b73d047
--- /dev/null
+++ b/package/gettext-tiny/gettext-tiny.hash
@@ -0,0 +1,2 @@
+# Locally Computed:
+sha256 40a003e850ae8c29b8e6e6321925596c3a57d7ae8296d880dc1e88a07aebcd93  gettext-tiny-f733dd3fdd7be973f523a464165aae827a17d838.tar.gz
diff --git a/package/gettext-tiny/gettext-tiny.mk b/package/gettext-tiny/gettext-tiny.mk
new file mode 100644
index 0000000000..d24f181c0b
--- /dev/null
+++ b/package/gettext-tiny/gettext-tiny.mk
@@ -0,0 +1,38 @@
+################################################################################
+#
+# gettext-tiny
+#
+################################################################################
+
+GETTEXT_TINY_VERSION = f733dd3fdd7be973f523a464165aae827a17d838
+GETTEXT_TINY_SITE = $(call github,sabotage-linux,gettext-tiny,$(GETTEXT_TINY_VERSION))
+GETTEXT_TINY_LICENSE = MIT
+GETTEXT_TINY_INSTALL_STAGING = YES
+GETTEXT_TINY_LICENSE_FILES = LICENSE
+
+ifeq ($(BR2_TOOLCHAIN_USES_MUSL),y)
+LIBINTL=MUSL
+endif
+
+ifeq ($(BR2_ENABLE_LOCALE),)
+GETTEXT_TINY_DEPENDENCIES = libiconv
+endif
+
+define GETTEXT_TINY_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) \
+		LIBINTL=$(LIBINTL) \
+		$(TARGET_CONFIGURE_OPTS)
+endef
+
+define GETTEXT_TINY_INSTALL_STAGING_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) \
+		$(TARGET_CONFIGURE_OPTS) \
+		DESTDIR=$(STAGING_DIR) prefix=/usr install
+endef
+
+define GETTEXT_TINY_INSTALL_TARGET_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(TARGET_CONFIGURE_OPTS) \
+		DESTDIR=$(TARGET_DIR) prefix=/usr install
+endef
+
+$(eval $(generic-package))
-- 
2.14.1

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Baruch Siach]

Hi Vadim,

Vadim Kochan writes:

> From: Vadim Kochan <vadim.kochan@petcube.com>
>
> Add gettext-tiny package from the sabotage-linux project:
>
> gettext-tiny provides lightweight replacements for tools typically used
> from the GNU gettext suite, which is incredibly bloated and takes a lot
> of time to build (in the order of an hour on slow devices). the most
> notable component is msgfmt which is used to create binary translation
> files in the .mo format out of textual input files in .po format. this
> is the most important tool for building software from source, because it
> is used from the build processes of many software packages.

Thanks. gettext is indeed a huge CPU cycles drain.

> Signed-off-by: Vadim Kochan <vadim.kochan@petcube.com>

[...]

> +GETTEXT_TINY_VERSION = f733dd3fdd7be973f523a464165aae827a17d838
> +GETTEXT_TINY_SITE = $(call github,sabotage-linux,gettext-tiny,$(GETTEXT_TINY_VERSION))
> +GETTEXT_TINY_LICENSE = MIT
> +GETTEXT_TINY_INSTALL_STAGING = YES
> +GETTEXT_TINY_LICENSE_FILES = LICENSE
> +
> +ifeq ($(BR2_TOOLCHAIN_USES_MUSL),y)
> +LIBINTL=MUSL
> +endif
> +
> +ifeq ($(BR2_ENABLE_LOCALE),)
> +GETTEXT_TINY_DEPENDENCIES = libiconv
> +endif
> +
> +define GETTEXT_TINY_BUILD_CMDS
> +	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) \
> +		LIBINTL=$(LIBINTL) \
> +		$(TARGET_CONFIGURE_OPTS)
> +endef
> +
> +define GETTEXT_TINY_INSTALL_STAGING_CMDS
> +	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) \
> +		$(TARGET_CONFIGURE_OPTS) \
> +		DESTDIR=$(STAGING_DIR) prefix=/usr install
> +endef
> +
> +define GETTEXT_TINY_INSTALL_TARGET_CMDS
> +	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(TARGET_CONFIGURE_OPTS) \
> +		DESTDIR=$(TARGET_DIR) prefix=/usr install
> +endef
> +
> +$(eval $(generic-package))

What about a host package? Would it be feasible to replace host-gettext
with host-gettext-tiny?

baruch

--
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Baruch Siach]

Hi Vadim,

Vadim Kochan writes:

> 2018-08-21 17:31 GMT+03:00 Baruch Siach <baruch@tkos.co.il>:
>> Vadim Kochan writes:
>>> From: Vadim Kochan <vadim.kochan@petcube.com>
>>> Add gettext-tiny package from the sabotage-linux project:
>>>
>>> gettext-tiny provides lightweight replacements for tools typically used
>>> from the GNU gettext suite, which is incredibly bloated and takes a lot
>>> of time to build (in the order of an hour on slow devices). the most
>>> notable component is msgfmt which is used to create binary translation
>>> files in the .mo format out of textual input files in .po format. this
>>> is the most important tool for building software from source, because it
>>> is used from the build processes of many software packages.
>>
>> Thanks. gettext is indeed a huge CPU cycles drain.
>>
>>> Signed-off-by: Vadim Kochan <vadim.kochan@petcube.com>
>>
>> [...]
>>
>>> +GETTEXT_TINY_VERSION = f733dd3fdd7be973f523a464165aae827a17d838
>>> +GETTEXT_TINY_SITE = $(call github,sabotage-linux,gettext-tiny,$(GETTEXT_TINY_VERSION))
>>> +GETTEXT_TINY_LICENSE = MIT
>>> +GETTEXT_TINY_INSTALL_STAGING = YES
>>> +GETTEXT_TINY_LICENSE_FILES = LICENSE
>>> +
>>> +ifeq ($(BR2_TOOLCHAIN_USES_MUSL),y)
>>> +LIBINTL=MUSL
>>> +endif
>>> +
>>> +ifeq ($(BR2_ENABLE_LOCALE),)
>>> +GETTEXT_TINY_DEPENDENCIES = libiconv
>>> +endif
>>> +
>>> +define GETTEXT_TINY_BUILD_CMDS
>>> +     $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) \
>>> +             LIBINTL=$(LIBINTL) \
>>> +             $(TARGET_CONFIGURE_OPTS)
>>> +endef
>>> +
>>> +define GETTEXT_TINY_INSTALL_STAGING_CMDS
>>> +     $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) \
>>> +             $(TARGET_CONFIGURE_OPTS) \
>>> +             DESTDIR=$(STAGING_DIR) prefix=/usr install
>>> +endef
>>> +
>>> +define GETTEXT_TINY_INSTALL_TARGET_CMDS
>>> +     $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(TARGET_CONFIGURE_OPTS) \
>>> +             DESTDIR=$(TARGET_DIR) prefix=/usr install
>>> +endef
>>> +
>>> +$(eval $(generic-package))
>>
>> What about a host package? Would it be feasible to replace host-gettext
>> with host-gettext-tiny?
>
> Well, I really do not know about usage it as host-getttext
> replacements (but I will try!),
> just add it for target only. Is it ok to let it go for the target only
> now and then
> check it if it can be used for the host mode ?

Of course. Adding a target gettext is perfectly fine as a first
step. Replacing host-gettext is a much bigger change, that can be done
later.

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Thomas Petazzoni]

Baruch, Vadim,

On Tue, 21 Aug 2018 17:31:20 +0300, Baruch Siach wrote:

> > gettext-tiny provides lightweight replacements for tools typically used
> > from the GNU gettext suite, which is incredibly bloated and takes a lot
> > of time to build (in the order of an hour on slow devices). the most
> > notable component is msgfmt which is used to create binary translation
> > files in the .mo format out of textual input files in .po format. this
> > is the most important tool for building software from source, because it
> > is used from the build processes of many software packages.  
> 
> Thanks. gettext is indeed a huge CPU cycles drain.

Agreed, I was looking forward to someone taking over the task of
packaging gettext-tiny. So thanks Vadim for taking on this task!

However, packaging gettext-tiny requires a lot more than just creating
a package for it. Indeed, how do you select whether you want to use
gettext or gettext-tiny ? There should be some global knob that defines
that, and which affects the TARGET_NLS_DEPENDENCIES variable.

And then, there's the case of packages that need gettext even when NLS
support is disabled.

So: a big YES to adding gettext-tiny in Buildroot, but we want a proper
clean integration, not just a separate package. Vadim: you can read
https://buildroot.org/downloads/manual/manual.html#_gettext_integration_and_interaction_with_packages
to learn a bit about the current gettext integration in Buildroot.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Vadim Kochan]

On Tue, Aug 21, 2018 at 6:29 PM Thomas Petazzoni <
thomas.petazzoni@bootlin.com> wrote:

> Baruch, Vadim,
>
> On Tue, 21 Aug 2018 17:31:20 +0300, Baruch Siach wrote:
>
> > > gettext-tiny provides lightweight replacements for tools typically used
> > > from the GNU gettext suite, which is incredibly bloated and takes a lot
> > > of time to build (in the order of an hour on slow devices). the most
> > > notable component is msgfmt which is used to create binary translation
> > > files in the .mo format out of textual input files in .po format. this
> > > is the most important tool for building software from source, because
> it
> > > is used from the build processes of many software packages.
> >
> > Thanks. gettext is indeed a huge CPU cycles drain.
>
> Agreed, I was looking forward to someone taking over the task of
> packaging gettext-tiny. So thanks Vadim for taking on this task!
>
> However, packaging gettext-tiny requires a lot more than just creating
> a package for it. Indeed, how do you select whether you want to use
> gettext or gettext-tiny ? There should be some global knob that defines
> that, and which affects the TARGET_NLS_DEPENDENCIES variable.
>
> And then, there's the case of packages that need gettext even when NLS
> support is disabled.
>
> So: a big YES to adding gettext-tiny in Buildroot, but we want a proper
> clean integration, not just a separate package. Vadim: you can read
>
> https://buildroot.org/downloads/manual/manual.html#_gettext_integration_and_interaction_with_packages
> to learn a bit about the current gettext integration in Buildroot.
>
> Thanks!
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> Embedded Linux and Kernel engineering
> https://bootlin.com


OK!

I got the point, so it is not really useful to add gettext-tiny just as a
package but integrate it
as gettext alternative. So let me clarify - you expect to see this at least
as series:
1) add package
2) integrate it as gettext alternative

rather than just separate package.

Thank you,
Vadim Kochan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180821/a9b9ad02/attachment.html>

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Thomas Petazzoni]

Hello,

On Tue, 21 Aug 2018 18:51:00 +0300, Vadim Kochan wrote:

> I got the point, so it is not really useful to add gettext-tiny just as a
> package but integrate it
> as gettext alternative. So let me clarify - you expect to see this at least
> as series:
> 1) add package
> 2) integrate it as gettext alternative

Yes, exactly. And as Baruch suggested, it would be excellent if we
could also use host-gettext-tiny as an alternative to host-gettext.

This will raise a few questions:

 - There are a few packages that unconditionally need host-gettext. Is
   this really true ?

 - Some packages need host-gettext because of the AM_ICONV m4 macro
   provided by gettext.m4. Is gettext-tiny also providing the
   gettext-related m4 macros ?

Overall, I believe it's not an easy task and it may take some time, but
it is definitely worth the effort, and I'll be happy to support this
effort.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Vadim Kochan]

Hi,

On Tue, Aug 21, 2018 at 8:36 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello,
>
> On Tue, 21 Aug 2018 18:51:00 +0300, Vadim Kochan wrote:
>
> > I got the point, so it is not really useful to add gettext-tiny just as a
> > package but integrate it
> > as gettext alternative. So let me clarify - you expect to see this at least
> > as series:
> > 1) add package
> > 2) integrate it as gettext alternative
>
> Yes, exactly. And as Baruch suggested, it would be excellent if we
> could also use host-gettext-tiny as an alternative to host-gettext.
>
> This will raise a few questions:
>
>  - There are a few packages that unconditionally need host-gettext. Is
>    this really true ?
Not sure about this)

>
>  - Some packages need host-gettext because of the AM_ICONV m4 macro
>    provided by gettext.m4. Is gettext-tiny also providing the
>    gettext-related m4 macros ?
Yes it has some set of m4 macrosses, but it needs to be tested how it
works for the buildroot.

What if I will start from converting gettext package into the virtual
one (with renaming the original
gettext -> gettext-gnu)  ? And after I will try to add gettext-tiny as
gettext provider.

Regards,
Vadim Kochan

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Thomas Petazzoni]

Hello Vadim,

On Tue, 21 Aug 2018 23:07:59 +0300, Vadim Kochan wrote:

> >  - There are a few packages that unconditionally need host-gettext. Is
> >    this really true ?  
> Not sure about this)

In this situation, we have:

 - aumix
 - host-guile
 - host-intltool (perhaps legit ?)
 - host-libglib2
 - libv4l (for AM_ICONV m4 macro)
 - linphone
 - mediastreamer
 - host-mono
 - sysstat (which also uses TARGET_NLS_DEPENDENCIES... weird)
 - tvheadend
 - host-wine
 - xkeyboard-config
 - zbar (for AM_ICONV m4 macro)

> >  - Some packages need host-gettext because of the AM_ICONV m4 macro
> >    provided by gettext.m4. Is gettext-tiny also providing the
> >    gettext-related m4 macros ?  
> Yes it has some set of m4 macrosses, but it needs to be tested how it
> works for the buildroot.
> 
> What if I will start from converting gettext package into the virtual
> one (with renaming the original
> gettext -> gettext-gnu)  ? And after I will try to add gettext-tiny as
> gettext provider.

Looks OK to me. I think the choice between gettext and gettext-tiny is
really a top-level decision, so it probably belongs near the "Enable
Native Language Support (NLS)" option in "System configuration".

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Vadim Kochan]

Hello!

On Tue, Aug 21, 2018 at 11:10 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello Vadim,
>
> On Tue, 21 Aug 2018 23:07:59 +0300, Vadim Kochan wrote:
>
> > >  - There are a few packages that unconditionally need host-gettext. Is
> > >    this really true ?
> > Not sure about this)
>
> In this situation, we have:
>
>  - aumix
>  - host-guile
>  - host-intltool (perhaps legit ?)
>  - host-libglib2
>  - libv4l (for AM_ICONV m4 macro)
>  - linphone
>  - mediastreamer
>  - host-mono
>  - sysstat (which also uses TARGET_NLS_DEPENDENCIES... weird)
>  - tvheadend
>  - host-wine
>  - xkeyboard-config
>  - zbar (for AM_ICONV m4 macro)

Meanwhile I did convert gettext -> gettext-gnu and made gettext as
virtual package
which selects gettext-gnu by default. I tested compilation of
libglib2, sysstat, zbar, libv4l
and linphone together with force selected gettext-tiny (as gettext provider)
and the compilation was success.

Regards,
Vadim Kochan

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Thomas Petazzoni]

Hello Vadim,

On Tue, 21 Aug 2018 17:22:26 +0300, Vadim Kochan wrote:
> From: Vadim Kochan <vadim.kochan@petcube.com>
> 
> Add gettext-tiny package from the sabotage-linux project:
> 
> gettext-tiny provides lightweight replacements for tools typically used
> from the GNU gettext suite, which is incredibly bloated and takes a lot
> of time to build (in the order of an hour on slow devices). the most
> notable component is msgfmt which is used to create binary translation
> files in the .mo format out of textual input files in .po format. this
> is the most important tool for building software from source, because it
> is used from the build processes of many software packages.
> 
> Signed-off-by: Vadim Kochan <vadim.kochan@petcube.com>

Have you been able to make further progress on packaging gettext-tiny
as an alternative to the full gettext ? This is definitely something
I'd like to see integrated in Buildroot, so let us know if you are
still working or not, or if someone else needs to step up/help.

Thanks a lot,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Vadim Kochan]

On Tue, Oct 9, 2018 at 3:45 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello Vadim,
>
> On Tue, 21 Aug 2018 17:22:26 +0300, Vadim Kochan wrote:
> > From: Vadim Kochan <vadim.kochan@petcube.com>
> >
> > Add gettext-tiny package from the sabotage-linux project:
> >
> > gettext-tiny provides lightweight replacements for tools typically used
> > from the GNU gettext suite, which is incredibly bloated and takes a lot
> > of time to build (in the order of an hour on slow devices). the most
> > notable component is msgfmt which is used to create binary translation
> > files in the .mo format out of textual input files in .po format. this
> > is the most important tool for building software from source, because it
> > is used from the build processes of many software packages.
> >
> > Signed-off-by: Vadim Kochan <vadim.kochan@petcube.com>
>
> Have you been able to make further progress on packaging gettext-tiny
> as an alternative to the full gettext ? This is definitely something
> I'd like to see integrated in Buildroot, so let us know if you are
> still working or not, or if someone else needs to step up/help.
>
> Thanks a lot,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

Hi Thomas!

I did turn gettext package into virtual one and renamed  gettext as
gettext-gnu and added gettext-tiny (so they became
as virtual providers). While gettext-gnu works ok, I am still trying
to resolve some issues with gettext-tiny. I will let you know
in few days if I have some progress.

Regards,
Vadim Kochan

[Buildroot] [PATCH 1/1] package/gettext-tiny: Add new package

[Thomas Petazzoni]

Hello,

On Tue, 9 Oct 2018 16:12:11 +0300, Vadim Kochan wrote:

> I did turn gettext package into virtual one and renamed  gettext as
> gettext-gnu and added gettext-tiny (so they became
> as virtual providers). While gettext-gnu works ok, I am still trying
> to resolve some issues with gettext-tiny. I will let you know
> in few days if I have some progress.

OK, great. Glad to hear you're still working on it. Looking forward to
seeing the patch series.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com