From mboxrd@z Thu Jan  1 00:00:00 1970
From: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Date: Tue, 31 Jul 2018 22:55:40 +0200
Subject: [Buildroot] [RFC PATCH] host-localedef: Compile against glibc
In-Reply-To: <20180730053936.26460-1-sam@mendozajonas.com>
References: <20180730053936.26460-1-sam@mendozajonas.com>
Message-ID: <20180731225540.3cd9fcd8@windsurf>
List-Id: <buildroot.busybox.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
To: buildroot@busybox.net

Hello Samuel,

Thanks for taking a look into this!

On Mon, 30 Jul 2018 15:39:36 +1000, Samuel Mendoza-Jonas wrote:
> In glibc 2.27 the following change occured:
> "Statically compiled applications attempting to load locales compiled
> for the GNU C Library version 2.27 will fail and fall back to the
> builtin C/POSIX locale."

I don't really understand this explanation. Isn't the problem that
glibc 2.27 doesn't read properly locale files generated by localedef
from glibc 2.24 ?

I kind of fail to parse the explanation given here :-/

> This impacts us since upstream buildroot uses a localdef built against
> an older eglibc release [0].
> Update the host-localedef package to build against glibc 2.27.
> Unfortunately we can't compile just the localedef utility, but even so
> this does not increase the build time too significantly. eg:
> 
> With glibc localedef:
> real    33m11.278s
> user    171m51.684s
> sys     19m46.528s
> 
> With eglibc localedef:
> real    31m12.715s
> user    155m13.616s
> sys     16m23.504s

Meh, two more minutes just to build some not so complicated user-space
program :-/

> -HOST_LOCALEDEF_CONF_ENV = CFLAGS="$(HOST_CFLAGS) -fgnu89-inline"
> +# glibc is part of the toolchain so disable the toolchain dependency
> +HOST_LOCALEDEF_ADD_TOOLCHAIN_DEPENDENCY = NO

Not needed for host-localedef.

> -# The makefile does not implement an install target
> +HOST_LOCALEDEF_SUBDIR = build
> +
> +# Thumb build is broken, build in ARM mode
> +ifeq ($(BR2_ARM_INSTRUCTIONS_THUMB),y)

This is a target related option...

> +HOST_LOCALEDEF_EXTRA_CFLAGS += -marm

... so it is not relevant to observe it when building a host program.

> +endif
> +
> +# MIPS64 defaults to n32 so pass the correct -mabi if
> +# we are using a different ABI. OABI32 is also used
> +# in MIPS so we pass -mabi=32 in this case as well
> +# even though it's not strictly necessary.
> +ifeq ($(BR2_MIPS_NABI64),y)
> +HOST_LOCALEDEF_EXTRA_CFLAGS += -mabi=64
> +else ifeq ($(BR2_MIPS_OABI32),y)
> +HOST_LOCALEDEF_EXTRA_CFLAGS += -mabi=32
> +endif

Same comment.

> +ifeq ($(BR2_ENABLE_DEBUG),y)
> +HOST_LOCALEDEF_EXTRA_CFLAGS += -g
> +endif

Same comment.

> +
> +# The stubs.h header is not installed by install-headers, but is
> +# needed for the gcc build. An empty stubs.h will work, as explained
> +# in http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick
> +# is used by Crosstool-NG.
> +ifeq ($(BR2_TOOLCHAIN_BUILDROOT_HOST_LOCALEDEF),y)
> +define HOST_LOCALEDEF_ADD_MISSING_STUB_H
> +	mkdir -p $(STAGING_DIR)/usr/include/gnu
> +	touch $(STAGING_DIR)/usr/include/gnu/stubs.h

STAGING_DIR is observed when building target packages, not host
packages, so this seems useless.

> +endef
> +endif
> +
> +# Even though we use the autotools-package infrastructure, we have to
> +# override the default configure commands for several reasons:
> +#
> +#  1. We have to build out-of-tree, but we can't use the same
> +#     'symbolic link to configure' used with the gcc packages.
> +#
> +#  2. We have to execute the configure script with bash and not sh.
> +#
> +# Note that as mentionned in
> +# http://patches.openembedded.org/patch/38849/, glibc must be
> +# built with -O2, so we pass our own CFLAGS and CXXFLAGS below.
> +define HOST_LOCALEDEF_CONFIGURE_CMDS
> +	mkdir -p $(@D)/build
> +	# Do the configuration
> +	(cd $(@D)/build; \
> +		$(HOST_CONFIGURE_OPTS) \
> +		CFLAGS="-O2 $(HOST_LOCALEDEF_EXTRA_CFLAGS)" CPPFLAGS="" \
> +		CXXFLAGS="-O2 $(HOST_LOCALEDEF_EXTRA_CFLAGS)" \
> +		$(SHELL) $(@D)/configure \
> +		ac_cv_path_BASH_SHELL=/bin/bash \
> +		libc_cv_forced_unwind=yes \
> +		libc_cv_ssp=no \
> +		--target=$(GNU_HOST_NAME) \
> +		--host=$(GNU_HOST_NAME) \
> +		--build=$(GNU_HOST_NAME) \
> +		--prefix=/usr \

Should be --prefix=$(HOST_DIR)

> +		$(if $(BR2_SOFT_FLOAT),--without-fp,--with-fp) \
> +		$(if $(BR2_x86_64),--enable-lock-elision) \

Both BR2_SOFT_FLOAT and BR2_x86_64 are target options, so observing
them when building a host package is almost always wrong.

> +		--with-pkgversion="Buildroot" \
> +		--without-cvs \
> +		--disable-profile \
> +		--without-gd \
> +		--enable-obsolete-rpc)
> +	$(HOST_LOCALEDEF_ADD_MISSING_STUB_H)
> +endef

With all this, aren't we building a full glibc for the host ?

Can you use the locale/others and locale/install-others make targets
instead ?

See how they are doing it in PTXdist:
https://git.pengutronix.de/cgit/ptxdist/tree/rules/host-localedef.make.
Most likely using the locale/others make target will help cut the build
time significantly.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com