[Buildroot] [PATCH v5 1/3] package/gobject-introspection: fix host-linking

[Buildroot] [PATCH v5 1/3] package/gobject-introspection: fix host-linking

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

When building, gobject-introspection uses tools/g-ir-scanner to build several
.gir and .typelib files. g-ir-scanner does not use LDFLAGS, and by default,
links to the system-installed libglib2 path. To remedy this issue, defining
either LD_LIBRARY_PATH or LD_RUN_PATH forces g-ir-scanner to use the host
installed libglib2 files. The decision to use LD_RUN_PATH instead of
LD_LIBRARY_PATH is because this is a run time issue with g-ir-scanner and not
a compile-time issue.

g-ir-scanner can use the argument --lib-dirs-envvar to pass a list of library
directories in place of LD_LIBRARY_PATH, which the installed g-ir-scanner
wrapper does. However, during the build process, this is not possible due to
the build process calling g-ir-scanner directly.

This fix has the added benefit of allowing the host gobject-introspection to
build the host .gir, .rnc, and .typelib files, which some packages may require.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
Changes v1 -> v3:
  - Update comment to be more clear.

Changes v3 -> v4:
  - Update patch to work with gobject-introspection 2.64.0

Changes v4 -> v5:
  - Changed LD_LIBRARY_PATH to LD_RUN_PATH after more testing.
  - Updated the commit message and comments in the .mk file to better describe
    what the problem was and why this is necessary.

 .../gobject-introspection/gobject-introspection.mk  | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/package/gobject-introspection/gobject-introspection.mk b/package/gobject-introspection/gobject-introspection.mk
index c18a1f17d3..19071ba1cb 100644
--- a/package/gobject-introspection/gobject-introspection.mk
+++ b/package/gobject-introspection/gobject-introspection.mk
@@ -33,11 +33,14 @@ HOST_GOBJECT_INTROSPECTION_DEPENDENCIES = \
 GOBJECT_INTROSPECTION_NINJA_ENV += \
 	CC="$(TARGET_CC)"
 
-# Disable introspection data on the host, as it is not needed and
-# the package will attempt to use the systems libglib2 which will fail
-# if the systems libglib2 version is older than 2.60.
-HOST_GOBJECT_INTROSPECTION_CONF_OPTS = \
-	-Dbuild_introspection_data=false
+# When building, gobject-introspection uses tools/g-ir-scanner to build several
+# .gir and .typelib files. g-ir-scanner does not use LDFLAGS, and by default,
+# links to the system-installed libglib2 path. To remedy this issue, defining
+# either LD_LIBRARY_PATH or LD_RUN_PATH forces g-ir-scanner to use the host
+# installed libglib2 files. Use LD_RUN_PATH instead of LD_LIBRARY_PATH because
+# this is a run time issue with g-ir-scanner and not a compile-time issue.
+HOST_GOBJECT_INTROSPECTION_NINJA_ENV += \
+	LD_RUN_PATH="$(HOST_DIR)/lib"
 
 # Use the host gi-scanner to prevent the scanner from generating incorrect
 # elf classes.
-- 
2.24.1

[Buildroot] [PATCH v5 2/3] package/meson: determine g-ir-scanner and g-ir-compiler paths from pkgconfig

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

Currently, meson hard codes the paths of these binaries which results in
cross-compiled environments to run the host versions of these tools.
However, GObject-introspection provides the appropriate paths to these
utilities via pkg-config

find_program is needed in the case g-i is built as a subproject. If
g-ir-scanner or g-ir-compiler are in the build or source directory use those.
If they aren't found in the source directory, use the results from pkg-config.

Current upstream-status: merged
https://github.com/mesonbuild/meson/pull/6687
https://github.com/mesonbuild/meson/pull/6696

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
Changes v1 -> v3:
  - Use an upstream patch that fixes this issue permanently.

Changes v4 -> v5:
  - Split the patches into two seperate patches. (Yann)

 ...canner-and-g-ir-compiler-paths-from-.patch | 61 +++++++++++++++++++
 ...nome.py-Fix-giscanner-and-gicompiler.patch | 40 ++++++++++++
 2 files changed, 101 insertions(+)
 create mode 100644 package/meson/0004-determine-g-ir-scanner-and-g-ir-compiler-paths-from-.patch
 create mode 100644 package/meson/0005-mesonbuild-modules-gnome.py-Fix-giscanner-and-gicompiler.patch

diff --git a/package/meson/0004-determine-g-ir-scanner-and-g-ir-compiler-paths-from-.patch b/package/meson/0004-determine-g-ir-scanner-and-g-ir-compiler-paths-from-.patch
new file mode 100644
index 0000000000..b0cdd3af1c
--- /dev/null
+++ b/package/meson/0004-determine-g-ir-scanner-and-g-ir-compiler-paths-from-.patch
@@ -0,0 +1,61 @@
+From f66b04b0996eae5cd7b0ad007435d5a51f28b691 Mon Sep 17 00:00:00 2001
+From: Adam Duskett <Aduskett@gmail.com>
+Date: Mon, 24 Feb 2020 06:29:26 -0800
+Subject: [PATCH] gobject-introspection: determine g-ir-scanner and
+ g-ir-compiler paths from pkgconfig
+
+Currently, meson hard codes the paths of these binaries which results in
+cross-compiled environments to run the host versions of these tools.
+However, GObject-introspection provides the appropriate paths to these
+utilities via pkg-config
+
+find_program is needed in the case g-i is built as a subproject. If
+g-ir-scanner or g-ir-compiler are in the build or source directory use those.
+If they aren't found in the source directory, use the results from pkg-config.
+
+Current upstream-status: merged
+https://github.com/mesonbuild/meson/pull/6687
+
+Signed-off-by: Adam Duskett <Aduskett@gmail.com>
+---
+ mesonbuild/modules/gnome.py | 21 ++++++++++++++++++---
+ 1 file changed, 18 insertions(+), 3 deletions(-)
+
+diff --git a/mesonbuild/modules/gnome.py b/mesonbuild/modules/gnome.py
+index 3d5d718136..0ea4b273e5 100644
+--- a/mesonbuild/modules/gnome.py
++++ b/mesonbuild/modules/gnome.py
+@@ -736,15 +736,30 @@ def generate_gir(self, state, args, kwargs):
+         if kwargs.get('install_dir'):
+             raise MesonException('install_dir is not supported with generate_gir(), see "install_dir_gir" and "install_dir_typelib"')
+ 
+-        giscanner = self.interpreter.find_program_impl('g-ir-scanner')
+-        gicompiler = self.interpreter.find_program_impl('g-ir-compiler')
+-
+         girtargets = [self._unwrap_gir_target(arg, state) for arg in args]
+ 
+         if len(girtargets) > 1 and any([isinstance(el, build.Executable) for el in girtargets]):
+             raise MesonException('generate_gir only accepts a single argument when one of the arguments is an executable')
+ 
+         self.gir_dep, pkgargs = self._get_gir_dep(state)
++        # find_program is needed in the case g-i is built as subproject.
++        # In that case it uses override_find_program so the gobject utilities
++        # can be used from the build dir instead of from the system.
++        # However, GObject-introspection provides the appropriate paths to
++        # these utilities via pkg-config, so it would be best to use the
++        # results from pkg-config when possible.
++        gi_util_dirs_check = [state.environment.get_build_dir(), state.environment.get_source_dir()]
++        giscanner = self.interpreter.find_program_impl('g-ir-scanner')
++        if giscanner.found():
++            giscanner_path = giscanner.get_command()[0]
++            if not any(x in giscanner_path for x in gi_util_dirs_check):
++                giscanner = self.gir_dep.get_pkgconfig_variable('g_ir_scanner', {})
++
++        gicompiler = self.interpreter.find_program_impl('g-ir-compiler')
++        if gicompiler.found():
++            gicompiler_path = gicompiler.get_command()[0]
++            if not any(x in gicompiler_path for x in gi_util_dirs_check):
++                gicompiler = self.gir_dep.get_pkgconfig_variable('g_ir_compiler', {})
+ 
+         ns = kwargs.pop('namespace')
+         nsversion = kwargs.pop('nsversion')
diff --git a/package/meson/0005-mesonbuild-modules-gnome.py-Fix-giscanner-and-gicompiler.patch b/package/meson/0005-mesonbuild-modules-gnome.py-Fix-giscanner-and-gicompiler.patch
new file mode 100644
index 0000000000..23fc461a95
--- /dev/null
+++ b/package/meson/0005-mesonbuild-modules-gnome.py-Fix-giscanner-and-gicompiler.patch
@@ -0,0 +1,40 @@
+From 1b059c974989ef36567454fa3b0034acb2db3691 Mon Sep 17 00:00:00 2001
+From: Adam Duskett <Aduskett@gmail.com>
+Date: Wed, 26 Feb 2020 05:51:28 -0800
+Subject: [PATCH] mesonbuild/modules/gnome.py: Fix giscanner and gicompiler
+ logic
+
+Currently, giscanner and the gicompiler paths are only scanned via pkg-config
+if they are first found in the host path.
+
+Add a else statement to fix this oversite.
+
+Current upstream-status: merged
+https://github.com/mesonbuild/meson/pull/6696
+
+Signed-off-by: Adam Duskett <Aduskett@gmail.com>
+---
+ mesonbuild/modules/gnome.py | 4 ++++
+ 1 file changed, 4 insertions(+)
+
+diff --git a/mesonbuild/modules/gnome.py b/mesonbuild/modules/gnome.py
+index 0ea4b273e5..1743b59c97 100644
+--- a/mesonbuild/modules/gnome.py
++++ b/mesonbuild/modules/gnome.py
+@@ -754,12 +754,16 @@ def generate_gir(self, state, args, kwargs):
+             giscanner_path = giscanner.get_command()[0]
+             if not any(x in giscanner_path for x in gi_util_dirs_check):
+                 giscanner = self.gir_dep.get_pkgconfig_variable('g_ir_scanner', {})
++        else:
++            giscanner = self.gir_dep.get_pkgconfig_variable('g_ir_scanner', {})
+ 
+         gicompiler = self.interpreter.find_program_impl('g-ir-compiler')
+         if gicompiler.found():
+             gicompiler_path = gicompiler.get_command()[0]
+             if not any(x in gicompiler_path for x in gi_util_dirs_check):
+                 gicompiler = self.gir_dep.get_pkgconfig_variable('g_ir_compiler', {})
++        else:
++            gicompiler = self.gir_dep.get_pkgconfig_variable('g_ir_compiler', {})
+ 
+         ns = kwargs.pop('namespace')
+         nsversion = kwargs.pop('nsversion')
-- 
2.24.1

[Buildroot] [PATCH v5 3/3] package/gobject-introspection: export variables in g-ir-scanner

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

When building .gir .typelib files, the g-ir-scanner wrapper calls the host
g-ir-scanner. g-ir-scanner calls ccompiler.py, which searches for the following
environment variables:
CC, CXX, CPP, LDFLAGS, CFLAGS, CPPFLAGS

These environment variables are empty by default, and as such ccompiler.py
defaults to either using the system tools (CC, CXX, and CPP) or leaving
the variables blank (LDFLAGS, CFLAGS, and CPPFLAGS.)

For autotools packages, this issue does not occur because autotools uses
Makefile.introspection found on the staging directory in
usr/share/gobject-introspection-1.0/ which automatically exports the above
variables.

However, for meson, the above variables are not exported when meson calls
g-ir-scanner to build .gir and .typelib files, which results in linking errors.

Exporting these variables in the g-ir-scanner wrapper fixes these issues and
ensures all build systems can generate .gir and .typelib files properly.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
Changes v1 -> v3:
  - Use relative paths. (Yann)

Changes v4 -> v5:
  - Switch back to using HOST_DIR. (Yann)
  - Update commit message as to why this patch is necessary.
  - Add CXX, CPP, LDFLAGS, CFLAGS and CPPFLAGS to the list of variables of
    which are exported.
  - Add a comment in g-ir-scanner as to why these exports are necessary.

 package/gobject-introspection/g-ir-scanner.in       | 13 +++++++++++++
 .../gobject-introspection/gobject-introspection.mk  |  9 +++++++++
 2 files changed, 22 insertions(+)

diff --git a/package/gobject-introspection/g-ir-scanner.in b/package/gobject-introspection/g-ir-scanner.in
index 9f97497b7e..13c99ce28f 100644
--- a/package/gobject-introspection/g-ir-scanner.in
+++ b/package/gobject-introspection/g-ir-scanner.in
@@ -1,6 +1,19 @@
 #!/usr/bin/env bash
 
+# These environment variables are empty by default, and as such ccompiler.py
+# defaults to either using the system tools (CC, CXX, and CPP) or leaving
+# the variables blank (LDFLAGS, CFLAGS, and CPPFLAGS.)
+# Export these variables in to ensure all build systems can generate .gir and
+# .typelib files properly.
+export CC="${HOST_DIR}/bin/@BASENAME_TARGET_CC@"
+export CXX="${HOST_DIR}/bin/@BASENAME_TARGET_CXX@"
+export CPP="${HOST_DIR}/bin/@BASENAME_TARGET_CPP@"
+export LDFLAGS="@TARGET_LDFLAGS@"
+export CFLAGS="@TARGET_CFLAGS@"
+export CPPFLAGS="@TARGET_CPPFLAGS@"
+
 export GI_SCANNER_DISABLE_CACHE=1
+
 ${HOST_DIR}/bin/g-ir-scanner \
 --lib-dirs-envvar=GIR_EXTRA_LIBS_PATH \
 --use-binary-wrapper=$(dirname $0)/g-ir-scanner-qemuwrapper \
diff --git a/package/gobject-introspection/gobject-introspection.mk b/package/gobject-introspection/gobject-introspection.mk
index 19071ba1cb..741cd27886 100644
--- a/package/gobject-introspection/gobject-introspection.mk
+++ b/package/gobject-introspection/gobject-introspection.mk
@@ -107,6 +107,15 @@ define GOBJECT_INTROSPECTION_INSTALL_WRAPPERS
 		$(INSTALL) -D -m 755 \
 			$(GOBJECT_INTROSPECTION_PKGDIR)/$(w).in $(STAGING_DIR)/usr/bin/$(w)
 	)
+	$(SED) "s%@BASENAME_TARGET_CC@%$(notdir $(TARGET_CC))%g" $(STAGING_DIR)/usr/bin/g-ir-scanner
+	$(SED) "s%@BASENAME_TARGET_CXX@%$(notdir $(TARGET_CXX))%g" $(STAGING_DIR)/usr/bin/g-ir-scanner
+	$(SED) "s%@BASENAME_TARGET_CPP@%$(notdir $(TARGET_CPP))%g" $(STAGING_DIR)/usr/bin/g-ir-scanner
+	$(SED) "s%@TARGET_LDFLAGS@%$(TARGET_LDFLAGS)%g" $(STAGING_DIR)/usr/bin/g-ir-scanner
+	$(SED) "s%@TARGET_CFLAGS@%$(TARGET_CFLAGS)%g" $(STAGING_DIR)/usr/bin/g-ir-scanner
+	$(SED) "s%@TARGET_CPPFLAGS@%$(TARGET_CPPFLAGS)%g" $(STAGING_DIR)/usr/bin/g-ir-scanner
+
+
+
 	# Gobject-introspection installs Makefile.introspection in
 	# $(STAGING_DIR)/usr/share which is needed for autotools-based programs to
 	# build .gir and .typelib files. Unfortuantly, gobject-introspection-1.0.pc
-- 
2.24.1

[Buildroot] [PATCH v5 1/3] package/gobject-introspection: fix host-linking

[Yann E. MORIN]

Adam, All,

On 2020-03-14 20:13 -0700, aduskett at gmail.com spake thusly:
> From: Adam Duskett <Aduskett@gmail.com>
> 
> When building, gobject-introspection uses tools/g-ir-scanner to build several
> .gir and .typelib files. g-ir-scanner does not use LDFLAGS, and by default,
> links to the system-installed libglib2 path. To remedy this issue, defining
> either LD_LIBRARY_PATH or LD_RUN_PATH forces g-ir-scanner to use the host
> installed libglib2 files. The decision to use LD_RUN_PATH instead of
> LD_LIBRARY_PATH is because this is a run time issue with g-ir-scanner and not
> a compile-time issue.
> 
> g-ir-scanner can use the argument --lib-dirs-envvar to pass a list of library
> directories in place of LD_LIBRARY_PATH, which the installed g-ir-scanner
> wrapper does. However, during the build process, this is not possible due to
> the build process calling g-ir-scanner directly.
> 
> This fix has the added benefit of allowing the host gobject-introspection to
> build the host .gir, .rnc, and .typelib files, which some packages may require.

So, this commit log is indeed better at exaplaining the problem and the
fix/workaround that is needed.

However, I still think it could be improved, so I respun a new iteration
of the series, with a few fixes of my own:
    https://patchwork.ozlabs.org/project/buildroot/list/?series=164476

Regards,
Yann E. MORIN.

> Signed-off-by: Adam Duskett <Aduskett@gmail.com>
> ---
> Changes v1 -> v3:
>   - Update comment to be more clear.
> 
> Changes v3 -> v4:
>   - Update patch to work with gobject-introspection 2.64.0
> 
> Changes v4 -> v5:
>   - Changed LD_LIBRARY_PATH to LD_RUN_PATH after more testing.
>   - Updated the commit message and comments in the .mk file to better describe
>     what the problem was and why this is necessary.
> 
>  .../gobject-introspection/gobject-introspection.mk  | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/package/gobject-introspection/gobject-introspection.mk b/package/gobject-introspection/gobject-introspection.mk
> index c18a1f17d3..19071ba1cb 100644
> --- a/package/gobject-introspection/gobject-introspection.mk
> +++ b/package/gobject-introspection/gobject-introspection.mk
> @@ -33,11 +33,14 @@ HOST_GOBJECT_INTROSPECTION_DEPENDENCIES = \
>  GOBJECT_INTROSPECTION_NINJA_ENV += \
>  	CC="$(TARGET_CC)"
>  
> -# Disable introspection data on the host, as it is not needed and
> -# the package will attempt to use the systems libglib2 which will fail
> -# if the systems libglib2 version is older than 2.60.
> -HOST_GOBJECT_INTROSPECTION_CONF_OPTS = \
> -	-Dbuild_introspection_data=false
> +# When building, gobject-introspection uses tools/g-ir-scanner to build several
> +# .gir and .typelib files. g-ir-scanner does not use LDFLAGS, and by default,
> +# links to the system-installed libglib2 path. To remedy this issue, defining
> +# either LD_LIBRARY_PATH or LD_RUN_PATH forces g-ir-scanner to use the host
> +# installed libglib2 files. Use LD_RUN_PATH instead of LD_LIBRARY_PATH because
> +# this is a run time issue with g-ir-scanner and not a compile-time issue.
> +HOST_GOBJECT_INTROSPECTION_NINJA_ENV += \
> +	LD_RUN_PATH="$(HOST_DIR)/lib"
>  
>  # Use the host gi-scanner to prevent the scanner from generating incorrect
>  # elf classes.
> -- 
> 2.24.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'