[Buildroot] [PATCH v4 1/2] support/misc/toolchainfile.cmake.in: don't set PKG_CONFIG_SYSROOT_DIR

[Buildroot] [PATCH v4 1/2] support/misc/toolchainfile.cmake.in: don't set PKG_CONFIG_SYSROOT_DIR

[James Hilliard]

This doesn't appear to be required and seems to break for packages
using meson's pkgconfig.relocatable format.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
 support/misc/toolchainfile.cmake.in | 1 -
 1 file changed, 1 deletion(-)

diff --git a/support/misc/toolchainfile.cmake.in b/support/misc/toolchainfile.cmake.in
index 30c0e63bf2..bf1fff4c5a 100644
--- a/support/misc/toolchainfile.cmake.in
+++ b/support/misc/toolchainfile.cmake.in
@@ -91,7 +91,6 @@ endif()
 if(NOT DEFINED CMAKE_FIND_ROOT_PATH_MODE_INCLUDE)
   set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
 endif()
-set(ENV{PKG_CONFIG_SYSROOT_DIR} "${RELOCATED_HOST_DIR}/@@STAGING_SUBDIR@@")
 
 # This toolchain file can be used both inside and outside Buildroot.
 if(NOT DEFINED CMAKE_C_COMPILER)
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[James Hilliard]

As of version 0.54.0 meson has had the ability to build and install
packages rather than having to run ninja directly as before.

This will allow us to use features such as meson install tags in
the future which require meson to be used for the installation.

Note that we've removed FOO_NINJA_ENV which may cause issues
in the unlikely event an external package uses it.

We need to set --pkgconfig.relocatable so that pkg-config files are
not generated with absolute paths which cause issues with per-package
directories.

Note that we need to avoid setting HOST_DIR in paths when configuring
as these will get appended to the destdir during installation which
will result in incorrect paths.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
Changes v3 -> v4:
  - split out cmake fix
  - cleanup config vars
  - clearify commit message
Changes v2 -> v3:
  - set pkgconfig to relocatable
  - set cmake pkg-config prefix path
Changes v1 -> v2:
  - update docs
---
 docs/manual/adding-packages-meson.txt         |  4 +-
 .../gobject-introspection.mk                  |  4 +-
 package/pkg-meson.mk                          | 61 +++++++++++++------
 package/systemd/systemd.mk                    |  4 +-
 4 files changed, 49 insertions(+), 24 deletions(-)

diff --git a/docs/manual/adding-packages-meson.txt b/docs/manual/adding-packages-meson.txt
index 029c8c2488..c8aed65c2f 100644
--- a/docs/manual/adding-packages-meson.txt
+++ b/docs/manual/adding-packages-meson.txt
@@ -125,8 +125,8 @@ will therefore only use a few of them.
   +c_link_args+, +cpp_args+, +cpp_link_args+, +sys_root+, and
   +pkg_config_libdir+.
 
-* +FOO_NINJA_ENV+, to specify additional environment variables to pass to
-  +ninja+, meson companion tool in charge of the build operations. By default,
+* +FOO_MESON_ENV+, to specify additional environment variables to pass to
+  +meson+, meson tool in charge of the build/install operations. By default,
   empty.
 
 * +FOO_NINJA_OPTS+, to specify a space-separated list of targets to build. By
diff --git a/package/gobject-introspection/gobject-introspection.mk b/package/gobject-introspection/gobject-introspection.mk
index 41d64171a7..ea5100247d 100644
--- a/package/gobject-introspection/gobject-introspection.mk
+++ b/package/gobject-introspection/gobject-introspection.mk
@@ -30,14 +30,14 @@ HOST_GOBJECT_INTROSPECTION_DEPENDENCIES = \
 	host-python3
 
 # g-ir-scanner will default to /usr/bin/ld for linking if this is not set.
-GOBJECT_INTROSPECTION_NINJA_ENV += \
+GOBJECT_INTROSPECTION_MESON_ENV += \
 	CC="$(TARGET_CC)"
 
 # When building, gobject-introspection uses tools/g-ir-scanner to build several
 # .gir and .typelib files. g-ir-scanner does not use LDFLAGS, and by default,
 # links to the system-installed libglib2 path. To remedy this issue, defining
 # LD_LIBRARY_PATH forces g-ir-scanner to use our host installed libglib2 files.
-HOST_GOBJECT_INTROSPECTION_NINJA_ENV += \
+HOST_GOBJECT_INTROSPECTION_MESON_ENV += \
 	LD_LIBRARY_PATH="$(if $(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)$(HOST_DIR)/lib"
 
 # Use the host gi-scanner to prevent the scanner from generating incorrect
diff --git a/package/pkg-meson.mk b/package/pkg-meson.mk
index 0835e08e3a..1123e3b904 100644
--- a/package/pkg-meson.mk
+++ b/package/pkg-meson.mk
@@ -21,13 +21,13 @@
 ################################################################################
 
 #
-# Pass PYTHONNOUSERSITE environment variable when invoking Meson or Ninja, so
+# Pass PYTHONNOUSERSITE environment variable when invoking Meson, so
 # $(HOST_DIR)/bin/python3 will not look for Meson modules in
 # $HOME/.local/lib/python3.x/site-packages
 #
-MESON		= PYTHONNOUSERSITE=y $(HOST_DIR)/bin/meson
-NINJA		= PYTHONNOUSERSITE=y $(HOST_DIR)/bin/ninja
-NINJA_OPTS	= $(if $(VERBOSE),-v)
+MESON              = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/meson
+MESON_BUILD_OPTS   = $(if $(VERBOSE),-v)
+MESON_INSTALL_OPTS = --no-rebuild
 
 # https://mesonbuild.com/Reference-tables.html#cpu-families
 ifeq ($(BR2_arcle)$(BR2_arceb),y)
@@ -136,6 +136,7 @@ define $(2)_CONFIGURE_CMDS
 	CXX_FOR_BUILD="$$(HOSTCXX)" \
 	$$($$(PKG)_CONF_ENV) \
 	$$(MESON) \
+		--pkgconfig.relocatable \
 		--prefix=/usr \
 		--libdir=lib \
 		--default-library=$(if $(BR2_STATIC_LIBS),static,shared) \
@@ -156,10 +157,11 @@ define $(2)_CONFIGURE_CMDS
 	mkdir -p $$($$(PKG)_SRCDIR)/build
 	$$(HOST_CONFIGURE_OPTS) \
 	$$($$(PKG)_CONF_ENV) $$(MESON) \
-		--prefix=$$(HOST_DIR) \
+		--pkgconfig.relocatable \
+		--prefix=/ \
 		--libdir=lib \
-		--sysconfdir=$$(HOST_DIR)/etc \
-		--localstatedir=$$(HOST_DIR)/var \
+		--sysconfdir=etc \
+		--localstatedir=var \
 		--default-library=shared \
 		--buildtype=release \
 		--wrap-mode=nodownload \
@@ -179,13 +181,23 @@ $(2)_DEPENDENCIES += host-meson
 ifndef $(2)_BUILD_CMDS
 ifeq ($(4),target)
 define $(2)_BUILD_CMDS
-	$$(TARGET_MAKE_ENV) $$($$(PKG)_NINJA_ENV) \
-		$$(NINJA) $$(NINJA_OPTS) $$($$(PKG)_NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build
+	$$(TARGET_MAKE_ENV) \
+	$$($$(PKG)_MESON_ENV) \
+	$$(MESON) \
+		compile \
+		$$(MESON_BUILD_OPTS) \
+		$$(if $$($$(PKG)_NINJA_OPTS),--ninja-args $$($$(PKG)_NINJA_OPTS)) \
+		-C $$($$(PKG)_SRCDIR)/build
 endef
 else
 define $(2)_BUILD_CMDS
-	$$(HOST_MAKE_ENV) $$($$(PKG)_NINJA_ENV) \
-		$$(NINJA) $$(NINJA_OPTS) $$($$(PKG)_NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build
+	$$(HOST_MAKE_ENV) \
+	$$($$(PKG)_MESON_ENV) \
+	$$(MESON) \
+		compile \
+		$$(MESON_BUILD_OPTS) \
+		$$(if $$($$(PKG)_NINJA_OPTS),--ninja-args $$($$(PKG)_NINJA_OPTS)) \
+		-C $$($$(PKG)_SRCDIR)/build
 endef
 endif
 endif
@@ -196,8 +208,13 @@ endif
 #
 ifndef $(2)_INSTALL_CMDS
 define $(2)_INSTALL_CMDS
-	$$(HOST_MAKE_ENV) $$($$(PKG)_NINJA_ENV) \
-		$$(NINJA) $$(NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build install
+	$$(HOST_MAKE_ENV) \
+	$$($$(PKG)_MESON_ENV) \
+	$$(MESON) \
+		install \
+		$$(MESON_INSTALL_OPTS) \
+		--destdir $$(HOST_DIR) \
+		-C $$($$(PKG)_SRCDIR)/build
 endef
 endif
 
@@ -207,8 +224,13 @@ endif
 #
 ifndef $(2)_INSTALL_STAGING_CMDS
 define $(2)_INSTALL_STAGING_CMDS
-	$$(TARGET_MAKE_ENV) $$($$(PKG)_NINJA_ENV) DESTDIR=$$(STAGING_DIR) \
-		$$(NINJA) $$(NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build install
+	$$(TARGET_MAKE_ENV) \
+	$$($$(PKG)_MESON_ENV) \
+	$$(MESON) \
+		install \
+		$$(MESON_INSTALL_OPTS) \
+		--destdir $$(STAGING_DIR) \
+		-C $$($$(PKG)_SRCDIR)/build
 endef
 endif
 
@@ -218,8 +240,13 @@ endif
 #
 ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
-	$$(TARGET_MAKE_ENV) $$($$(PKG)_NINJA_ENV) DESTDIR=$$(TARGET_DIR) \
-		$$(NINJA) $$(NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build install
+	$$(TARGET_MAKE_ENV) \
+	$$($$(PKG)_MESON_ENV) \
+	$$(MESON) \
+		install \
+		$$(MESON_INSTALL_OPTS) \
+		--destdir $$(TARGET_DIR) \
+		-C $$($$(PKG)_SRCDIR)/build
 endef
 endif
 
diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
index 47aaddf849..42d53422aa 100644
--- a/package/systemd/systemd.mk
+++ b/package/systemd/systemd.mk
@@ -749,7 +749,7 @@ endef
 SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SYSTEMD_PRESET_ALL
 
 SYSTEMD_CONF_ENV = $(HOST_UTF8_LOCALE_ENV)
-SYSTEMD_NINJA_ENV = $(HOST_UTF8_LOCALE_ENV)
+SYSTEMD_MESON_ENV = $(HOST_UTF8_LOCALE_ENV)
 
 define SYSTEMD_LINUX_CONFIG_FIXUPS
 	$(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS)
@@ -852,8 +852,6 @@ HOST_SYSTEMD_DEPENDENCIES = \
 	host-gperf \
 	host-python-jinja2
 
-HOST_SYSTEMD_NINJA_ENV = DESTDIR=$(HOST_DIR)
-
 # Fix RPATH After installation
 # * systemd provides a install_rpath instruction to meson because the binaries
 #   need to link with libsystemd which is not in a standard path
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[Thomas Petazzoni via buildroot]

Hello James,

On Mon, 25 Jul 2022 22:44:25 -0600
James Hilliard <james.hilliard1@gmail.com> wrote:

> Note that we need to avoid setting HOST_DIR in paths when configuring
> as these will get appended to the destdir during installation which
> will result in incorrect paths.

I *think* this is wrong, at least in the classical autotools sense of
prefix vs. DESTDIR.

> @@ -156,10 +157,11 @@ define $(2)_CONFIGURE_CMDS
>  	mkdir -p $$($$(PKG)_SRCDIR)/build
>  	$$(HOST_CONFIGURE_OPTS) \
>  	$$($$(PKG)_CONF_ENV) $$(MESON) \
> -		--prefix=$$(HOST_DIR) \

	--prefix=$$(HOST_DIR)

should stay here.

>  ifndef $(2)_INSTALL_CMDS
>  define $(2)_INSTALL_CMDS
> -	$$(HOST_MAKE_ENV) $$($$(PKG)_NINJA_ENV) \
> -		$$(NINJA) $$(NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build install
> +	$$(HOST_MAKE_ENV) \
> +	$$($$(PKG)_MESON_ENV) \
> +	$$(MESON) \
> +		install \
> +		$$(MESON_INSTALL_OPTS) \
> +		--destdir $$(HOST_DIR) \

And you should drop --destdir $$(HOST_DIR) here.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[James Hilliard]

On Tue, Jul 26, 2022 at 12:25 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> Hello James,
>
> On Mon, 25 Jul 2022 22:44:25 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > Note that we need to avoid setting HOST_DIR in paths when configuring
> > as these will get appended to the destdir during installation which
> > will result in incorrect paths.
>
> I *think* this is wrong, at least in the classical autotools sense of
> prefix vs. DESTDIR.

This isn't really different from how we handle target/staging prefix+install.

>
> > @@ -156,10 +157,11 @@ define $(2)_CONFIGURE_CMDS
> >       mkdir -p $$($$(PKG)_SRCDIR)/build
> >       $$(HOST_CONFIGURE_OPTS) \
> >       $$($$(PKG)_CONF_ENV) $$(MESON) \
> > -             --prefix=$$(HOST_DIR) \
>
>         --prefix=$$(HOST_DIR)
>
> should stay here.
>
> >  ifndef $(2)_INSTALL_CMDS
> >  define $(2)_INSTALL_CMDS
> > -     $$(HOST_MAKE_ENV) $$($$(PKG)_NINJA_ENV) \
> > -             $$(NINJA) $$(NINJA_OPTS) -C $$($$(PKG)_SRCDIR)/build install
> > +     $$(HOST_MAKE_ENV) \
> > +     $$($$(PKG)_MESON_ENV) \
> > +     $$(MESON) \
> > +             install \
> > +             $$(MESON_INSTALL_OPTS) \
> > +             --destdir $$(HOST_DIR) \
>
> And you should drop --destdir $$(HOST_DIR) here.

I recall this didn't work due to pkg-config file generation being prefix based
which was resulting in pkg-config files being incompatible with per-package
directories.

>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[Thomas Petazzoni via buildroot]

On Tue, 26 Jul 2022 00:38:11 -0600
James Hilliard <james.hilliard1@gmail.com> wrote:

> > I *think* this is wrong, at least in the classical autotools sense of
> > prefix vs. DESTDIR.  
> 
> This isn't really different from how we handle target/staging prefix+install.

It is completely different. Do you know the difference between prefix
and DESTDIR?

prefix is the path where the software will be located when executed.

DESTDIR is here to divert the installation.

For host packages, we use prefix=$(HOST_DIR) and no DESTDIR because the
software will be located in $(HOST_DIR) when executed.

For target packages, we use prefix=/usr and DESTDIR=$(TARGET_DIR) /
$(STAGING_DIR) because the software will be located in /usr when
executed on the target, but temporarily during the installation we need
the files to go in $(TARGET_DIR) or $(STAGING_DIR).

See also https://www.gnu.org/prep/standards/html_node/DESTDIR.html.

> > And you should drop --destdir $$(HOST_DIR) here.  
> 
> I recall this didn't work due to pkg-config file generation being prefix based
> which was resulting in pkg-config files being incompatible with per-package
> directories.

And this needs to be investigated since as explained above --prefix /
and --destdir $(HOST_DIR) is most likely incorrect for host packages.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[James Hilliard]

On Tue, Jul 26, 2022 at 1:07 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Tue, 26 Jul 2022 00:38:11 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > > I *think* this is wrong, at least in the classical autotools sense of
> > > prefix vs. DESTDIR.
> >
> > This isn't really different from how we handle target/staging prefix+install.
>
> It is completely different. Do you know the difference between prefix
> and DESTDIR?
>
> prefix is the path where the software will be located when executed.

We install to the package's per-package directory $(HOST_DIR) but
we don't actually execute it from there AFAIU.

Since the execution location changes for each per-package build I thought
we wanted the prefix to be independent of the absolute installation path.

>
> DESTDIR is here to divert the installation.
>
> For host packages, we use prefix=$(HOST_DIR) and no DESTDIR because the
> software will be located in $(HOST_DIR) when executed.

But the $(HOST_DIR) it's installed to is not the same $(HOST_DIR) it's executed
from when BR2_PER_PACKAGE_DIRECTORIES is enabled, at least when
it's executed from a reverse dependency.

>
> For target packages, we use prefix=/usr and DESTDIR=$(TARGET_DIR) /
> $(STAGING_DIR) because the software will be located in /usr when
> executed on the target, but temporarily during the installation we need
> the files to go in $(TARGET_DIR) or $(STAGING_DIR).
>
> See also https://www.gnu.org/prep/standards/html_node/DESTDIR.html.
>
> > > And you should drop --destdir $$(HOST_DIR) here.
> >
> > I recall this didn't work due to pkg-config file generation being prefix based
> > which was resulting in pkg-config files being incompatible with per-package
> > directories.
>
> And this needs to be investigated since as explained above --prefix /
> and --destdir $(HOST_DIR) is most likely incorrect for host packages.
>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[Thomas Petazzoni via buildroot]

On Tue, 26 Jul 2022 11:26:50 -0600
James Hilliard <james.hilliard1@gmail.com> wrote:

> > prefix is the path where the software will be located when executed.  
> 
> We install to the package's per-package directory $(HOST_DIR) but
> we don't actually execute it from there AFAIU.
> 
> Since the execution location changes for each per-package build I thought
> we wanted the prefix to be independent of the absolute installation path.
> 
> >
> > DESTDIR is here to divert the installation.
> >
> > For host packages, we use prefix=$(HOST_DIR) and no DESTDIR because the
> > software will be located in $(HOST_DIR) when executed.  
> 
> But the $(HOST_DIR) it's installed to is not the same $(HOST_DIR) it's executed
> from when BR2_PER_PACKAGE_DIRECTORIES is enabled, at least when
> it's executed from a reverse dependency.

That is indeed true, but this usage of prefix/DESTDIR is also what we
do in CMake, autotools, and everywhere else, and it did work fine until
now with Meson. I wouldn't want Meson to stand out from
CMake/autotools, unless we have a solid reason to do so. It's not clear
why switching to calling meson to do the build/install requires this
change.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v4 2/2] package/pkg-meson: use meson to build/install packages

[James Hilliard]

On Tue, Jul 26, 2022 at 11:34 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Tue, 26 Jul 2022 11:26:50 -0600
> James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > > prefix is the path where the software will be located when executed.
> >
> > We install to the package's per-package directory $(HOST_DIR) but
> > we don't actually execute it from there AFAIU.
> >
> > Since the execution location changes for each per-package build I thought
> > we wanted the prefix to be independent of the absolute installation path.
> >
> > >
> > > DESTDIR is here to divert the installation.
> > >
> > > For host packages, we use prefix=$(HOST_DIR) and no DESTDIR because the
> > > software will be located in $(HOST_DIR) when executed.
> >
> > But the $(HOST_DIR) it's installed to is not the same $(HOST_DIR) it's executed
> > from when BR2_PER_PACKAGE_DIRECTORIES is enabled, at least when
> > it's executed from a reverse dependency.
>
> That is indeed true, but this usage of prefix/DESTDIR is also what we
> do in CMake, autotools, and everywhere else, and it did work fine until
> now with Meson. I wouldn't want Meson to stand out from
> CMake/autotools, unless we have a solid reason to do so. It's not clear
> why switching to calling meson to do the build/install requires this
> change.

Seems to work if I make sure all the prefix based paths are under $(HOST_DIR):
https://patchwork.ozlabs.org/project/buildroot/patch/20220727071557.3434892-2-james.hilliard1@gmail.com/

>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot