[Buildroot] [Bug 15181] New: Building libcurl fails due to faulty package definition

[Buildroot] [Bug 15181] New: Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

            Bug ID: 15181
           Summary: Building libcurl fails due to faulty package
                    definition
           Product: buildroot
           Version: 2022.02.8
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned@buildroot.uclibc.org
          Reporter: dmrauh@posteo.de
                CC: buildroot@uclibc.org
  Target Milestone: ---

Hi all,

I'm trying to compile "libcurl" using Buildroot 2022.02.8 with the following
relevant options in my .config:

BR2_PACKAGE_LIBCURL=y
BR2_PACKAGE_LIBCURL_OPENSSL=y
BR2_PACKAGE_OPENSSL=y


Unfortunately the build is reproducibly failing during "libcurl-configure" with
the following symptoms:

...
configure: PKG_CONFIG_LIBDIR will be set to
"/home/rauhdomi/build/out/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/pkgconfig"
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lcrypto "
configure: pkg-config: SSL_LDFLAGS: "-L/usr/lib64 "
configure: pkg-config: SSL_CPPFLAGS: "-I/usr/include "
checking for HMAC_Update in -lcrypto... no
checking for HMAC_Init_ex in -lcrypto... no
checking OpenSSL linking with -ldl... no
checking OpenSSL linking with -ldl and -lpthread... no
configure: OPT_OPENSSL:
/home/rauhdomi/build/out/host/x86_64-buildroot-linux-gnu/sysroot/usr
configure: OPENSSL_ENABLED: 
configure: error: --with-openssl was given but OpenSSL could not be detected
make[1]: *** [package/pkg-generic.mk:283:
/home/rauhdomi/build/out/build/libcurl-7.84.0/.stamp_configured] Error 1
make: *** [Makefile:40: _all] Error 2

After poking around for a bit I narrowed it down to the following part of
libcurl's package definition in Buildroot "libcurl.mk":

# configure adds the cross openssl dir to LD_LIBRARY_PATH which screws up
# native stuff during the rest of configure when target == host.
# Fix it by setting LD_LIBRARY_PATH to something sensible so those libs
# are found first.
LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if
$(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)/lib:/usr/lib

Since on my build system "pkgconf" is also installed through my distro's
package manager (in version 1.8.0), its "libpkgconf.so.3" is used instead of
the one part of "$(HOST_DIR)/lib", when building "libcurl". Apparently this has
never been a problem before because the versions were semantically compatible,
which isn't the case any longer.

I guess this bug has the potential to affect any user having "pkgconf" already
installed on their system.

My proposal on how to fix the bug would be as follows:

diff --git a/package/libcurl/libcurl.mk b/package/libcurl/libcurl.mk
index e241bd1c88..f6f5501f83 100644
--- a/package/libcurl/libcurl.mk
+++ b/package/libcurl/libcurl.mk
@@ -46,7 +46,7 @@ LIBCURL_DEPENDENCIES += openssl
 # native stuff during the rest of configure when target == host.
 # Fix it by setting LD_LIBRARY_PATH to something sensible so those libs
 # are found first.
-LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if
$(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)/lib:/usr/lib
+LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if
$(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)$(HOST_DIR)/lib:/lib:/usr/lib
 LIBCURL_CONF_OPTS += --with-ssl=$(STAGING_DIR)/usr \
        --with-ca-path=/etc/ssl/certs
 else

This way we always make sure that libs part of the SDK generated by Buildroot
take precedence over the ones coming from the build system.

Best regards,
Dominik

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [Bug 15181] Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

--- Comment #1 from Thomas Petazzoni <thomas.petazzoni@bootlin.com> ---
Thanks for the great investigation.

One question is whether the LD_LIBRARY_PATH trick is still needed? It was added
back in 2009, so perhaps things have changed in libcurl?

This commit in libcurl:

commit 2d4c2152c9eb3dbdf943de46ed8fc11285f1b90b
Author: Daniel Stenberg <daniel@haxx.se>
Date:   Fri Apr 13 14:07:39 2018 +0200

seems to have reworked how this works, including for openssl.

Also, I have removed the LD_LIBRARY_PATH tricked, and built the following
configuration on an x86-64 machine (to have the target == host case):

BR2_x86_64=y
BR2_x86_corei7=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_X86_64_CORE_I7_GLIBC_BLEEDING_EDGE=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_OPENSSL=y
BR2_PACKAGE_LIBCURL=y
# BR2_TARGET_ROOTFS_TAR is not set

and it built just fine.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [Bug 15181] Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

--- Comment #2 from Jookia <contact@jookia.org> ---
My friend Xogium hit this issue this month and after a few debugging sessions
we tracked this down. Here's what I've found:

- Arch Linux's host toolchain now sets runpath instead of rpath
- libcurl sets LD_LIBRARY_PATH to /lib:/usr/lib
- Buildroot pkgconf and Arch Linux pkgconf versions line up
- Arch Linux pkgconf library is installed to /usr/lib/libpkgconf.so
- Buildroot pkgconf uses the host libpkgconf.so due to LD_LIBRARY_PATH
- this is respected because rpath does not override it
- The Buildroot pkg-conf script sets LIBPATH to build/host/bin/../sysroot/...
and SYSROOT to build/host/sysroot
- pkgconf 1.8 silently ignores LIBPATH if it's outside sysroot
- pkgconf 1.8 does not normalize paths
- pkgconf reverted LIBPATH to the one in the host's libpkg.so, /lib and
/usr/lib
- curl build tries to find openssl but can't

Two fixes are needed:

- libcurl build should not set LD_LIBRARY_PATH
- the pkg-config wrapper should use normalized paths

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [Bug 15181] Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |yann.morin.1998@free.fr
             Status|NEW                         |RESOLVED
         Resolution|---                         |MOVED

--- Comment #3 from Yann E. MORIN <yann.morin.1998@free.fr> ---
Thank you for your report.

The issue tracker for the Buildroot project has been moved to
the Gitlab.com issue tracker:
    https://gitlab.com/buildroot.org/buildroot/-/issues

We are taking this opportunity to close old issues in this old
tracker. If you believe your issue is still relevant, please
open one in the new issue tracker.

Thank you!

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot