[Buildroot] [PATCH] package/qemu: force the host compiler to look at HOST_DIR first

[Buildroot] [PATCH] package/qemu: force the host compiler to look at HOST_DIR first

[Romain Naour]

Since meson 1.2.0, compiler's search path are retrieved and used
to link a library. If the same library is installed on the host
and build by Buildroot (HOST_DIR/lib), meson now link against
the one found in the compiler's search path instead of linking
agains the one provided by Buildroot.

See: https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566

Help the compiler by setting -Bprefix that specifies a path where to find the
executables, libraries, include files, and data files of the compiler itself.
For each subprogram to be run, the compiler driver first tries the -B prefix,
if any. If that name is not found, or if -B is not specified, the driver
tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.

Fixes:
    host/lib/libgio-2.0.so: undefined reference to `g_module_open_full'
    collect2: error: ld returned 1 exit statu

The problem only appear if we have two version of the glib2 library installed
on the system:

[Buildroot]/host/lib/libgio-2.0.so.0.7600.1
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8

This is probably a meson bug introduced by [1] since the error message is wrong!

$ strings output/host/lib/libgio-2.0.so | grep g_module_open_full
g_module_open_full

The libgio-2.0.so provided by Buildroot contains the g_module_open_full symbol
while the libgio-2.0.so installed on the host is too old to have it:

$ strings /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8 | grep g_module_open_full

[1] https://github.com/mesonbuild/meson/commit/59cfbf68e00aa774a9868101f423bd662938c15d

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Nicolas CARRIER <carrier.nicolas0@gmail.com>
---
This meson issue may be not specific to host-qemu...
---
 package/qemu/qemu.mk | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index b341d0220d..8a8dddee58 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -401,7 +401,18 @@ HOST_QEMU_ARCH = sh4eb
 endif
 HOST_QEMU_SYS_ARCH ?= $(HOST_QEMU_ARCH)
 
-HOST_QEMU_CFLAGS = $(HOST_CFLAGS)
+# Since meson 1.2.0, compiler's search path are retrieved and used
+# to link a library. If the same library is installed on the host
+# and build by Buildroot (HOST_DIR/lib), meson now link against
+# the one found in the compiler's search path instead of linking
+# agains the one provided by Buildroot.
+# https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
+# Help the compiler by setting -Bprefix that specifies where to find the
+# executables, libraries, include files, and data files of the compiler itself.
+# For each subprogram to be run, the compiler driver first tries the -B prefix,
+# if any. If that name is not found, or if -B is not specified, the driver
+# tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
+HOST_QEMU_CFLAGS = $(HOST_CFLAGS) -B$(HOST_DIR)/lib
 
 ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
 HOST_QEMU_TARGETS += $(HOST_QEMU_SYS_ARCH)-softmmu
-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] package/qemu: force the host compiler to look at HOST_DIR first

[Yann E. MORIN]

Romain, All,

+Bernd for the meson bump

On 2023-08-19 01:46 +0200, Romain Naour spake thusly:
> Since meson 1.2.0, compiler's search path are retrieved and used
> to link a library. If the same library is installed on the host
> and build by Buildroot (HOST_DIR/lib), meson now link against
> the one found in the compiler's search path instead of linking
> agains the one provided by Buildroot.
> 
> See: https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
> 
> Help the compiler by setting -Bprefix that specifies a path where to find the
> executables, libraries, include files, and data files of the compiler itself.
> For each subprogram to be run, the compiler driver first tries the -B prefix,
> if any. If that name is not found, or if -B is not specified, the driver
> tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.

So, I understand that this makes qemu actually build, but I am afraid
this is just hidign the problem.

Indeed, it is possible that we have other host packages that depend on
some of the host libs we build and that may exist on the system, and
those packages are also going to break.

For example, host-efl is a host-meson-package, and depends on
host-libglib2, so presumably we would hav e the issue there too.

We can't fix the breakage by adding -B to all the host packages one by
one. If -B is idneed the correct fix (I doubt it), then we will have to
add it to the common HOST_CFLAGS, HOST_CXXFLAGS and so on...

However, what bothers me slighhtly, is why the qemu link step uses two
different ways to link with the libglib2 libraries, e.g.:

    -l/path/to/host/lib/libgio-2.0.so
    -L/path/to/host/lib -lgmodule-2.0

So, why is libgio handled with an absolute path, while libgmodule is
not? Note that libgio does have a DT_NEEDED on libgmodule.

However, let's have a look at the pkg-config files, let's start with
gio-2.0.pc:

    Requires: glib-2.0, gobject-2.0
    Requires.private: gmodule-no-export-2.0, zlib, mount >= 2.23
    Libs: -L${libdir} -lgio-2.0
    Libs.private: -ldl -lresolv

OK, then gmodule-no-export-2.0.pc:

    Requires: glib-2.0
    Libs: -L${libdir} -lgmodule-2.0 -pthread
    Libs.private: -ldl

So, the two are similar. I am not sure to understand how we end up in
this situation, though... :-/

In the meantime, since the bump to meson 1.2.0 is causing the issue with
out an easy fix in sight, I'll revert the meson bump.

Regards,
Yann E. MORIN.

> Fixes:
>     host/lib/libgio-2.0.so: undefined reference to `g_module_open_full'
>     collect2: error: ld returned 1 exit statu
> 
> The problem only appear if we have two version of the glib2 library installed
> on the system:
> 
> [Buildroot]/host/lib/libgio-2.0.so.0.7600.1
> /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8
> 
> This is probably a meson bug introduced by [1] since the error message is wrong!
> 
> $ strings output/host/lib/libgio-2.0.so | grep g_module_open_full
> g_module_open_full
> 
> The libgio-2.0.so provided by Buildroot contains the g_module_open_full symbol
> while the libgio-2.0.so installed on the host is too old to have it:
> 
> $ strings /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8 | grep g_module_open_full
> 
> [1] https://github.com/mesonbuild/meson/commit/59cfbf68e00aa774a9868101f423bd662938c15d
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> Cc: Nicolas CARRIER <carrier.nicolas0@gmail.com>
> ---
> This meson issue may be not specific to host-qemu...
> ---
>  package/qemu/qemu.mk | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
> index b341d0220d..8a8dddee58 100644
> --- a/package/qemu/qemu.mk
> +++ b/package/qemu/qemu.mk
> @@ -401,7 +401,18 @@ HOST_QEMU_ARCH = sh4eb
>  endif
>  HOST_QEMU_SYS_ARCH ?= $(HOST_QEMU_ARCH)
>  
> -HOST_QEMU_CFLAGS = $(HOST_CFLAGS)
> +# Since meson 1.2.0, compiler's search path are retrieved and used
> +# to link a library. If the same library is installed on the host
> +# and build by Buildroot (HOST_DIR/lib), meson now link against
> +# the one found in the compiler's search path instead of linking
> +# agains the one provided by Buildroot.
> +# https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
> +# Help the compiler by setting -Bprefix that specifies where to find the
> +# executables, libraries, include files, and data files of the compiler itself.
> +# For each subprogram to be run, the compiler driver first tries the -B prefix,
> +# if any. If that name is not found, or if -B is not specified, the driver
> +# tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
> +HOST_QEMU_CFLAGS = $(HOST_CFLAGS) -B$(HOST_DIR)/lib
>  
>  ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
>  HOST_QEMU_TARGETS += $(HOST_QEMU_SYS_ARCH)-softmmu
> -- 
> 2.41.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] package/qemu: force the host compiler to look at HOST_DIR first

[Romain Naour]

[-- Attachment #1.1: Type: text/plain, Size: 6462 bytes --]

Hello,

Le sam. 19 août 2023, 15:54, Yann E. MORIN <yann.morin.1998@free.fr> a
écrit :

> Romain, All,
>
> +Bernd for the meson bump
>
> On 2023-08-19 01:46 +0200, Romain Naour spake thusly:
> > Since meson 1.2.0, compiler's search path are retrieved and used
> > to link a library. If the same library is installed on the host
> > and build by Buildroot (HOST_DIR/lib), meson now link against
> > the one found in the compiler's search path instead of linking
> > agains the one provided by Buildroot.
> >
> > See:
> https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
> >
> > Help the compiler by setting -Bprefix that specifies a path where to
> find the
> > executables, libraries, include files, and data files of the compiler
> itself.
> > For each subprogram to be run, the compiler driver first tries the -B
> prefix,
> > if any. If that name is not found, or if -B is not specified, the driver
> > tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
>
> So, I understand that this makes qemu actually build, but I am afraid
> this is just hidign the problem.
>
> Indeed, it is possible that we have other host packages that depend on
> some of the host libs we build and that may exist on the system, and
> those packages are also going to break.
>
> For example, host-efl is a host-meson-package, and depends on
> host-libglib2, so presumably we would hav e the issue there too.
>
> We can't fix the breakage by adding -B to all the host packages one by
> one. If -B is idneed the correct fix (I doubt it), then we will have to
> add it to the common HOST_CFLAGS, HOST_CXXFLAGS and so on...
>

I'm not convinced either by this patch (see the comment bellow the ---
sign). Indeed host-qemu may not the only package having such issue...

>
> However, what bothers me slighhtly, is why the qemu link step uses two
> different ways to link with the libglib2 libraries, e.g.:
>
>     -l/path/to/host/lib/libgio-2.0.so
>     -L/path/to/host/lib -lgmodule-2.0
>
> So, why is libgio handled with an absolute path, while libgmodule is
> not? Note that libgio does have a DT_NEEDED on libgmodule.
>
> However, let's have a look at the pkg-config files, let's start with
> gio-2.0.pc:
>
>     Requires: glib-2.0, gobject-2.0
>     Requires.private: gmodule-no-export-2.0, zlib, mount >= 2.23
>     Libs: -L${libdir} -lgio-2.0
>     Libs.private: -ldl -lresolv
>
> OK, then gmodule-no-export-2.0.pc:
>
>     Requires: glib-2.0
>     Libs: -L${libdir} -lgmodule-2.0 -pthread
>     Libs.private: -ldl
>
> So, the two are similar. I am not sure to understand how we end up in
> this situation, though... :-/
>

I'm not sure to understand your comment about pc files, this may be another
problem?

This issue could be reproduced with another library.

Do you know any buildsystem using compiler's default paths like this ?


> In the meantime, since the bump to meson 1.2.0 is causing the issue with
> out an easy fix in sight, I'll revert the meson bump.
>

I believe this is the safe solution for the upcoming release.

Best regards,
Romain


> Regards,
> Yann E. MORIN.
>
> > Fixes:
> >     host/lib/libgio-2.0.so: undefined reference to `g_module_open_full'
> >     collect2: error: ld returned 1 exit statu
> >
> > The problem only appear if we have two version of the glib2 library
> installed
> > on the system:
> >
> > [Buildroot]/host/lib/libgio-2.0.so.0.7600.1
> > /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8
> >
> > This is probably a meson bug introduced by [1] since the error message
> is wrong!
> >
> > $ strings output/host/lib/libgio-2.0.so | grep g_module_open_full
> > g_module_open_full
> >
> > The libgio-2.0.so provided by Buildroot contains the g_module_open_full
> symbol
> > while the libgio-2.0.so installed on the host is too old to have it:
> >
> > $ strings /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8 | grep
> g_module_open_full
> >
> > [1]
> https://github.com/mesonbuild/meson/commit/59cfbf68e00aa774a9868101f423bd662938c15d
> >
> > Signed-off-by: Romain Naour <romain.naour@gmail.com>
> > Cc: Nicolas CARRIER <carrier.nicolas0@gmail.com>
> > ---
> > This meson issue may be not specific to host-qemu...
> > ---
> >  package/qemu/qemu.mk | 13 ++++++++++++-
> >  1 file changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
> > index b341d0220d..8a8dddee58 100644
> > --- a/package/qemu/qemu.mk
> > +++ b/package/qemu/qemu.mk
> > @@ -401,7 +401,18 @@ HOST_QEMU_ARCH = sh4eb
> >  endif
> >  HOST_QEMU_SYS_ARCH ?= $(HOST_QEMU_ARCH)
> >
> > -HOST_QEMU_CFLAGS = $(HOST_CFLAGS)
> > +# Since meson 1.2.0, compiler's search path are retrieved and used
> > +# to link a library. If the same library is installed on the host
> > +# and build by Buildroot (HOST_DIR/lib), meson now link against
> > +# the one found in the compiler's search path instead of linking
> > +# agains the one provided by Buildroot.
> > +#
> https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
> > +# Help the compiler by setting -Bprefix that specifies where to find the
> > +# executables, libraries, include files, and data files of the compiler
> itself.
> > +# For each subprogram to be run, the compiler driver first tries the -B
> prefix,
> > +# if any. If that name is not found, or if -B is not specified, the
> driver
> > +# tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
> > +HOST_QEMU_CFLAGS = $(HOST_CFLAGS) -B$(HOST_DIR)/lib
> >
> >  ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
> >  HOST_QEMU_TARGETS += $(HOST_QEMU_SYS_ARCH)-softmmu
> > --
> > 2.41.0
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
> --
>
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
> conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
>      |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is
> no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
>  conspiracy.  |
>
> '------------------------------^-------^------------------^--------------------'
>

[-- Attachment #1.2: Type: text/html, Size: 9781 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot