Re: [Buildroot] [PATCH v2 5/5] package/xz: convert to cmake build

[Peter Seiderer via buildroot <buildroot@buildroot.org>]

Hello Yann, *,

On Thu, 27 Jun 2024 13:16:20 +0200, yann.morin@orange.com wrote:

> Peter, All,
> 
> On 2024-06-27 10:26 +0200, Peter Seiderer spake thusly:
> > On Thu, 27 Jun 2024 09:50:29 +0200, Peter Seiderer <ps.report@gmx.net> wrote:  
> > > > Anyway, I have now reverted that patch, as it was exposing another issue
> > > > with host-python3 (https://gitlab.com/buildroot.org/buildroot/-/issues/5)  
> [--SNIP--]
> > The following quick patch solves the host-xz and host-python3 build failures
> > for me:
> > 
> > diff --git a/package/xz/xz.mk b/package/xz/xz.mk
> > index b22b96849d..b22ccac85f 100644
> > --- a/package/xz/xz.mk
> > +++ b/package/xz/xz.mk
> > @@ -12,7 +12,7 @@ XZ_LICENSE = Public Domain, BSD-0-Clause, GPL-2.0+, GPL-3.0+, LGPL-2.1+
> >  XZ_LICENSE_FILES = COPYING COPYING.0BSD COPYING.GPLv2 COPYING.GPLv3 COPYING.LGPLv2.1
> >  XZ_CPE_ID_VENDOR = tukaani
> > 
> > -XZ_CONF_OPTS = \
> > +XZ_COMMON_CONF_OPTS = \  
> 
> I think _COMMON_CONF_OPTS is to be avoided: it is easy to add/remove
> options from that list, that affects both builds, but we don't want to
> do that most of the time.
> 
> Looking at a few other packages, we leave the host options to a bare
> minimum, while there are more options specified for the target.
> 
> In this case, I see that he en/decoder list is indeed probably sharable,
> but I think I'd put it in a variable of its own:
> 
>     XZ_CODERS = lzma1;lzma2;delta;x86;powerpc;ia64;arm;armthumb;arm64;sparc;riscv
>     XZ_CONF_OPTS = \
>         -DENCODERS="$(XZ_CODERS)" \
>         -DDECODERS="$(XZ_CODERS)" \
>         ...
> 
>     HOST_XZ_CONF_OPTS = \
>         -DENCODERS="$(XZ_CODERS)" \
>         -DDECODERS="$(XZ_CODERS)" \
>         ...
> 
> But do we _have_ to pass that list at all? Can't we jsut tell xz to
> enable all of them? How can we detect that a coder is no longer valid
> and should be removed?
> 
> If I look at the repository
> 
>     https://github.com/tukaani-project/xz/blob/v5.6.2/CMakeLists.txt#L693
> 
>     set(ENCODERS "${SUPPORTED_FILTERS}" CACHE STRING "Encoders to support")
> 
> So it would seem the default is to enable everything, and we should just
> drop the option, no?
> 
> >  	-DENCODERS="lzma1;lzma2;delta;x86;powerpc;ia64;arm;armthumb;arm64;sparc;riscv" \
> >  	-DDECODERS="lzma1;lzma2;delta;x86;powerpc;ia64;arm;armthumb;arm64;sparc;riscv" \
> >  	-DMATCH_FINDERS="hc3;hc4;bt2;bt3;bt4" \
> > @@ -25,8 +25,9 @@ XZ_CONF_OPTS = \
> >  	-DENABLE_SMALL=OFF \
> >  	-DENABLE_SANDBOX=ON \
> >  	-DCREATE_XZ_SYMLINKS=ON \
> > -	-DCREATE_LZMA_SYMLINKS=ON \
> > -	-DBUILD_SHARED_LIBS=OFF
> > +	-DCREATE_LZMA_SYMLINKS=ON
> > +
> > +XZ_CONF_OPTS = $(XZ_COMMON_CONF_OPTS)
> > 
> >  ifeq ($(BR2_SYSTEM_ENABLE_NLS),y)
> >  XZ_CONF_OPTS += -DENABLE_NLS=ON
> > @@ -42,11 +43,11 @@ endif
> > 
> >  # we are built before ccache
> >  HOST_XZ_CONF_OPTS = \
> > -	$(XZ_CONF_OPTS) \
> > -	-DENABLE_NLS=ON \
> > +	$(XZ_COMMON_CONF_OPTS) \
> > +	-DENABLE_NLS=OFF \  
> 
> But the real question is: why does it sometines work, and sometimes not?
> This needs to be expained and traced in the commit log.

I believe it is the combination of '-DENABLE_NLS=ON' and the legacy cmake
version in the container (3.18.4), see CMakeLists.txt:

 290 if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.20")
 291     find_package(Intl)
 292     find_package(Gettext)
 293 
 294     if(Intl_FOUND)
 295         option(ENABLE_NLS "Native Language Support (translated messages)" ON)
[...]
 304         if(ENABLE_NLS AND NOT GETTEXT_FOUND AND
 305                 NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/po/uk.gmo")
 306             # This only sets the variable, not the cache variable!
 307             set(ENABLE_NLS OFF)
[...]
 332     endif()
 333 endif()

Issue (and suggested fix) reported upstream:

	https://github.com/tukaani-project/xz/issues/129
	https://github.com/tukaani-project/xz/pull/130

> 
> On my F39, and on Arnout's F40, as well as on your system, it was
> working. What changed in the reference build environment that causes the
> build to fial? On IRC, Arnout had a suggestion: "I guess you have
> libintl-dev or something installed there?"
> 
> In any case, I believe such a conversion can only be done once upstream
> conisders the cmake buildsystem complete and official, which it
> currently does not.

Changed recently, see 'CMake: Not experimental anymore'
(https://github.com/tukaani-project/xz/commit/5279828635a95abdef82e691fc4979d362780e63)

> 
> Thank you for testing on your side as well! 👍
> 
> In the meantime, I had started a test in the background, buildig those
> configurations:
> 
>   - glibc, no NLS
>   - glibc, NLS
>   - musl, no NLS
>   - musl, NLS
>   - uClibc-NG, no NLS
>   - uClibc-NG, NLS
> 
> All 6 combinations yield a configuration where xz (the target variant)
> builds successfully. So it seems NLS=ON/OFF works OK for the target.

Did you test on a local system or container (which cmake version(s))?

Regards,
Peter

> 
> Regards,
> Yann E. MORIN.
> 
> >  	-DENABLE_THREADS=ON \
> >  	-DCMAKE_C_COMPILER_LAUNCHER="" \
> > -	-DCMAKE_CXX_COMPILER_LAUNCHER==""
> > +	-DCMAKE_CXX_COMPILER_LAUNCHER=""
> > 
> >  $(eval $(cmake-package))
> >  $(eval $(host-cmake-package))
> > 
> > Regards,
> > Peter
> > 
> >   
> > >
> > > Maybe although reverting 'package/xz: explicitly specify all autoconf options'
> > > (and 'package/xz: enable year2038 option') as I used the same
> > > 'HOST_XZ_CONF_OPTS = $(XZ_CONF_OPTS)...' pattern here?
> > >
> > > Regards,
> > > Peter
> > >  
> > 
> > [...]
> >   
> 

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot