[Buildroot] [PATCH 0/2] Ninja, Meson: new build systems

[Buildroot] [PATCH 0/2] Ninja, Meson: new build systems

[Eric Le Bihan]

This small patch series provides two new build systems: Ninja and Meson.

Eric Le Bihan (2):
  ninja: new package
  python-meson: new package

 package/Config.in                              |  2 +
 package/ninja/Config.in                        | 11 ++++++
 package/ninja/ninja.hash                       |  1 +
 package/ninja/ninja.mk                         | 39 ++++++++++++++++++
 package/python-meson/Config.in                 |  9 +++++
 package/python-meson/cross-compilation.conf.in | 19 +++++++++
 package/python-meson/meson-exe-wrapper.in      |  3 ++
 package/python-meson/python-meson.hash         |  1 +
 package/python-meson/python-meson.mk           | 55 ++++++++++++++++++++++++++
 9 files changed, 140 insertions(+)
 create mode 100644 package/ninja/Config.in
 create mode 100644 package/ninja/ninja.hash
 create mode 100644 package/ninja/ninja.mk
 create mode 100644 package/python-meson/Config.in
 create mode 100644 package/python-meson/cross-compilation.conf.in
 create mode 100755 package/python-meson/meson-exe-wrapper.in
 create mode 100644 package/python-meson/python-meson.hash
 create mode 100644 package/python-meson/python-meson.mk

-- 
2.4.11

[Buildroot] [PATCH 1/2] ninja: new package

[Eric Le Bihan]

This new package provides Ninja, a small build system with a focus on
speed.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
---
 package/Config.in        |  1 +
 package/ninja/Config.in  | 11 +++++++++++
 package/ninja/ninja.hash |  1 +
 package/ninja/ninja.mk   | 39 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 52 insertions(+)
 create mode 100644 package/ninja/Config.in
 create mode 100644 package/ninja/ninja.hash
 create mode 100644 package/ninja/ninja.mk

diff --git a/package/Config.in b/package/Config.in
index 9f6e0d9..4146bf9 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -148,6 +148,7 @@ endif
 	source "package/jq/Config.in"
 	source "package/libtool/Config.in"
 	source "package/make/Config.in"
+	source "package/ninja/Config.in"
 if BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
 	source "package/patch/Config.in"
 endif
diff --git a/package/ninja/Config.in b/package/ninja/Config.in
new file mode 100644
index 0000000..a8eba2e
--- /dev/null
+++ b/package/ninja/Config.in
@@ -0,0 +1,11 @@
+config BR2_PACKAGE_NINJA
+	bool "ninja"
+	depends on BR2_INSTALL_LIBSTDCPP
+	help
+	  Ninja is a small build system with a focus on speed.
+
+	  https://ninja-build.org/
+
+comment "ninja needs a toolchain w/ C++"
+	depends on !BR2_INSTALL_LIBSTDCPP
+
diff --git a/package/ninja/ninja.hash b/package/ninja/ninja.hash
new file mode 100644
index 0000000..e5908f3
--- /dev/null
+++ b/package/ninja/ninja.hash
@@ -0,0 +1 @@
+none xxx ninja-v1.7.1.tar.gz
diff --git a/package/ninja/ninja.mk b/package/ninja/ninja.mk
new file mode 100644
index 0000000..ae9251c
--- /dev/null
+++ b/package/ninja/ninja.mk
@@ -0,0 +1,39 @@
+################################################################################
+#
+# ninja
+#
+################################################################################
+
+NINJA_VERSION = v1.7.1
+NINJA_SITE = $(call github,ninja-build,ninja,$(NINJA_VERSION))
+NINJA_LICENSE = Apache-2.0
+NINJA_LICENSE_FILES = COPYING
+
+NINJA_DEPENDENCIES = host-ninja
+
+NINJA_MAKE_OPTS = CXX="$(TARGET_CXX)" AR="$(TARGET_AR)"
+
+define NINJA_CONFIGURE_CMDS
+	(cd $(@D); $(NINJA_MAKE_OPTS) ./configure.py)
+endef
+
+define NINJA_BUILD_CMDS
+	(cd $(@D); $(HOST_DIR)/usr/bin/ninja)
+endef
+
+define NINJA_INSTALL_TARGET_CMDS
+	$(INSTALL) -m 0755 $(@D)/ninja $(TARGET_DIR)/usr/bin/ninja
+endef
+
+HOST_NINJA_DEPENDENCIES = host-python
+
+define HOST_NINJA_BUILD_CMDS
+	(cd $(@D); ./configure.py --bootstrap)
+endef
+
+define HOST_NINJA_INSTALL_CMDS
+	$(INSTALL) -m 0755 $(@D)/ninja $(HOST_DIR)/usr/bin/ninja
+endef
+
+$(eval $(generic-package))
+$(eval $(host-generic-package))
-- 
2.4.11

[Buildroot] [PATCH 2/2] python-meson: new package

[Eric Le Bihan]

This new package provides the Meson Build System, an open source build
system meant to be both extremely fast, and as user friendly as
possible.

More precisely, Meson creates configuration files for the Ninja build
system.

The host variant is also provided, which, besides building Meson,
generates a cross-compilation configuration file:
'$(HOST_DIR)/etc/meson/cross-compilation.conf'.

To be able to perform checks when cross-compiling, the host variant
depends on host-qemu.

To cross-compile a Meson-based project for the target, its package
should:

 - depend on host-python-meson
 - invoke the host variant of Meson with the
   *--cross-file=$(HOST_DIR)/etc/meson/cross-compilation.conf* option.
 - invoke host variant of Ninja to perform the actual build.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
---
 package/Config.in                              |  1 +
 package/python-meson/Config.in                 |  9 +++++
 package/python-meson/cross-compilation.conf.in | 19 +++++++++
 package/python-meson/meson-exe-wrapper.in      |  3 ++
 package/python-meson/python-meson.hash         |  1 +
 package/python-meson/python-meson.mk           | 55 ++++++++++++++++++++++++++
 6 files changed, 88 insertions(+)
 create mode 100644 package/python-meson/Config.in
 create mode 100644 package/python-meson/cross-compilation.conf.in
 create mode 100755 package/python-meson/meson-exe-wrapper.in
 create mode 100644 package/python-meson/python-meson.hash
 create mode 100644 package/python-meson/python-meson.mk

diff --git a/package/Config.in b/package/Config.in
index 4146bf9..b81954a 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -688,6 +688,7 @@ menu "External python modules"
 	source "package/python-markdown/Config.in"
 	source "package/python-markupsafe/Config.in"
 	source "package/python-meld3/Config.in"
+	source "package/python-meson/Config.in"
 	source "package/python-mistune/Config.in"
 	source "package/python-msgpack/Config.in"
 	source "package/python-netaddr/Config.in"
diff --git a/package/python-meson/Config.in b/package/python-meson/Config.in
new file mode 100644
index 0000000..b4e0529
--- /dev/null
+++ b/package/python-meson/Config.in
@@ -0,0 +1,9 @@
+config BR2_PACKAGE_PYTHON_MESON
+	bool "python-meson"
+	depends on BR2_PACKAGE_PYTHON3
+	select BR2_PACKAGE_NINJA
+	help
+	  Meson is an open source build system meant to be both extremely
+	  fast, and, even more importantly, as user friendly as possible.
+
+	  http://mesonbuild.com
diff --git a/package/python-meson/cross-compilation.conf.in b/package/python-meson/cross-compilation.conf.in
new file mode 100644
index 0000000..b3da294
--- /dev/null
+++ b/package/python-meson/cross-compilation.conf.in
@@ -0,0 +1,19 @@
+[host_machine]
+system = 'linux'
+cpu_family = 'x86'
+cpu = 'generic'
+endian = 'little'
+
+[target_machine]
+system = 'linux'
+cpu_family ='@TARGET_ARCH@'
+cpu = 'generic'
+endian = '@TARGET_ENDIAN@'
+
+[binaries]
+c = '@TARGET_CROSS at gcc'
+cpp = '@TARGET_CROSS at g++'
+ar = '@TARGET_CROSS at ar'
+strip = '@TARGET_CROSS at strip'
+exe_wrapper = '@HOST_DIR@/usr/bin/meson-exe-wrapper'
+pkgconfig = '@HOST_DIR@/usr/bin/pkg-config'
diff --git a/package/python-meson/meson-exe-wrapper.in b/package/python-meson/meson-exe-wrapper.in
new file mode 100755
index 0000000..5e4415c
--- /dev/null
+++ b/package/python-meson/meson-exe-wrapper.in
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+ at HOST_DIR@/usr/bin/qemu- at TARGET_ARCH@ -L @TARGET_DIR@ "$@"
diff --git a/package/python-meson/python-meson.hash b/package/python-meson/python-meson.hash
new file mode 100644
index 0000000..7af56c5
--- /dev/null
+++ b/package/python-meson/python-meson.hash
@@ -0,0 +1 @@
+none xxx python-meson-0.31.0.tar.gz
diff --git a/package/python-meson/python-meson.mk b/package/python-meson/python-meson.mk
new file mode 100644
index 0000000..d4ba4dd
--- /dev/null
+++ b/package/python-meson/python-meson.mk
@@ -0,0 +1,55 @@
+################################################################################
+#
+# python-meson
+#
+################################################################################
+
+PYTHON_MESON_VERSION = 0.31.0
+PYTHON_MESON_SITE = $(call github,mesonbuild,meson,$(PYTHON_MESON_VERSION))
+PYTHON_MESON_LICENSE = Apache-2.0
+PYTHON_MESON_LICENSE_FILES = COPYING
+PYTHON_MESON_SETUP_TYPE = setuptools
+
+define PYTHON_MESON_REMOVE_GUI_TOOL
+	rm -f $(TARGET_DIR)/usr/bin/mesongui.py
+endef
+
+PYTHON_MESON_POST_INSTALL_TARGET_HOOKS += PYTHON_MESON_REMOVE_GUI_TOOL
+
+HOST_PYTHON_MESON_DEPENDENCIES = host-qemu
+HOST_PYTHON_MESON_NEEDS_HOST_PYTHON = python3
+
+HOST_PYTHON_MESON_TARGET_ARCH = $(call qstrip,$(BR2_ARCH))
+HOST_PYTHON_MESON_TARGET_ENDIAN = $(shell echo $(BR2_ENDIAN) | tr 'A-Z' 'a-z')
+
+define HOST_PYTHON_MESON_REMOVE_GUI_TOOL
+	rm -f $(HOST_DIR)/usr/bin/mesongui.py
+endef
+
+define HOST_PYTHON_MESON_INSTALL_CROSS_CONF
+	$(INSTALL) -D -m 0644 package/python-meson/cross-compilation.conf.in \
+		$(HOST_DIR)/etc/meson/cross-compilation.conf
+	$(SED) 's;@TARGET_CROSS@;$(TARGET_CROSS);g' \
+		-e 's;@TARGET_ARCH@;$(HOST_PYTHON_MESON_TARGET_ARCH);g' \
+		-e 's;@TARGET_ENDIAN@;$(HOST_PYTHON_MESON_TARGET_ENDIAN);g' \
+		-e 's;@HOST_DIR@;$(HOST_DIR);g' \
+		$(HOST_DIR)/etc/meson/cross-compilation.conf
+endef
+
+define HOST_PYTHON_MESON_INSTALL_EXE_WRAPPER
+	$(INSTALL) -D -m 0755 package/python-meson/meson-exe-wrapper.in \
+		$(HOST_DIR)/usr/bin/meson-exe-wrapper
+	$(SED) 's;@TARGET_ARCH@;$(HOST_PYTHON_MESON_TARGET_ARCH);g' \
+		-e 's;@HOST_DIR@;$(HOST_DIR);g' \
+		-e 's;@TARGET_DIR@;$(TARGET_DIR);g' \
+		$(HOST_DIR)/usr/bin/meson-exe-wrapper
+endef
+
+HOST_PYTHON_MESON_POST_INSTALL_HOOKS += \
+	HOST_PYTHON_MESON_REMOVE_GUI_TOOL \
+	HOST_PYTHON_MESON_INSTALL_CROSS_CONF \
+	HOST_PYTHON_MESON_INSTALL_EXE_WRAPPER
+
+
+$(eval $(python-package))
+$(eval $(host-python-package))
-- 
2.4.11

[Buildroot] [PATCH 1/2] ninja: new package

[Bernd Kuhls]

Hi,

please try to add support for python3:

> +HOST_NINJA_DEPENDENCIES = host-python

HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)

> +define HOST_NINJA_BUILD_CMDS
> +	(cd $(@D); ./configure.py --bootstrap)
> +endef

define HOST_NINJA_BUILD_CMDS
	(cd $(@D); ./configure.py --bootstrap --verbose --with-python=$(HOST_DIR)/usr/bin/python)
endef

What is the purpose of having ninja on the target machine?

Regards, Bernd

[Buildroot] [PATCH 1/2] ninja: new package

[Eric Le Bihan]

Hi!

Le Sat, 04 Jun 2016 17:50:34 +0200,
Bernd Kuhls <bernd.kuhls@t-online.de> a ?crit :

> please try to add support for python3:

OK, will do!

> What is the purpose of having ninja on the target machine?

Ninja is used by Meson, which I also provide for the target. It is true
that the "Development tools" section now only lists "make" as build
system, since autotools have been removed. I do not know if Buildroot
users frequently need build systems on their target, but I provided it
anyway.

If the policy is to provide build system packages only for the host,
I'm fine with it.

Thanks for the comments.

-- 
ELB

[Buildroot] [PATCH 1/2] ninja: new package

[Thomas Petazzoni]

Hello,

On Sat,  4 Jun 2016 15:34:02 +0200, Eric Le Bihan wrote:

> diff --git a/package/ninja/ninja.hash b/package/ninja/ninja.hash
> new file mode 100644
> index 0000000..e5908f3
> --- /dev/null
> +++ b/package/ninja/ninja.hash
> @@ -0,0 +1 @@
> +none xxx ninja-v1.7.1.tar.gz

Please add a locally calculated hash here. Tarballs generated by github
are now stable, so we can add hashes.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/2] ninja: new package

[Thomas Petazzoni]

Hello,

On Sat, 4 Jun 2016 20:03:01 +0200, Eric Le Bihan wrote:

> Ninja is used by Meson, which I also provide for the target. It is true
> that the "Development tools" section now only lists "make" as build
> system, since autotools have been removed. I do not know if Buildroot
> users frequently need build systems on their target, but I provided it
> anyway.

"make" was only kept in the development tools because it was needed to
be able to *run* the uClibc test suite on the target. This has been
changed a few days ago, so I believe we don't really need "make"
anymore as a target package.

> If the policy is to provide build system packages only for the host,
> I'm fine with it.

Yes, please provide them only as host packages. However, do you have a
target package that would use them? Without this, the host packages are
orphan and never tested by anything, which isn't good.

So could you submit ninja/python-meson together with at least one
target package that uses them?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2] python-meson: new package

[Thomas Petazzoni]

Hello,

On Sat,  4 Jun 2016 15:34:03 +0200, Eric Le Bihan wrote:
> This new package provides the Meson Build System, an open source build
> system meant to be both extremely fast, and as user friendly as
> possible.
> 
> More precisely, Meson creates configuration files for the Ninja build
> system.
> 
> The host variant is also provided, which, besides building Meson,
> generates a cross-compilation configuration file:
> '$(HOST_DIR)/etc/meson/cross-compilation.conf'.
> 
> To be able to perform checks when cross-compiling, the host variant
> depends on host-qemu.
> 
> To cross-compile a Meson-based project for the target, its package
> should:
> 
>  - depend on host-python-meson
>  - invoke the host variant of Meson with the
>    *--cross-file=$(HOST_DIR)/etc/meson/cross-compilation.conf* option.
>  - invoke host variant of Ninja to perform the actual build.

It would probably be good to write this somewhere, as it is quite
important to know for users of this build system. But I'm not sure
where: you're going to have only a host package, so no Config.in file,
and therefore no help text. And I'm not sure where this could be added
in the Buildroot manual.

>  package/Config.in                              |  1 +
>  package/python-meson/Config.in                 |  9 +++++

As discussed, if it's a build system, please add only a host package.


> diff --git a/package/python-meson/python-meson.hash b/package/python-meson/python-meson.hash
> new file mode 100644
> index 0000000..7af56c5
> --- /dev/null
> +++ b/package/python-meson/python-meson.hash
> @@ -0,0 +1 @@
> +none xxx python-meson-0.31.0.tar.gz

Since you're fetching from Github, please add a real hash.


> +HOST_PYTHON_MESON_DEPENDENCIES = host-qemu
> +HOST_PYTHON_MESON_NEEDS_HOST_PYTHON = python3
> +
> +HOST_PYTHON_MESON_TARGET_ARCH = $(call qstrip,$(BR2_ARCH))

You can directly use the $(ARCH) variable, which is defined in the main
Makefile.

> +HOST_PYTHON_MESON_TARGET_ENDIAN = $(shell echo $(BR2_ENDIAN) | tr 'A-Z' 'a-z')

$(call LOWERCASE,$(call qstrip,$(BR2_ENDIAN))

> +
> +define HOST_PYTHON_MESON_REMOVE_GUI_TOOL
> +	rm -f $(HOST_DIR)/usr/bin/mesongui.py
> +endef

Not sure removing stuff from the host variant is really useful.

> +
> +define HOST_PYTHON_MESON_INSTALL_CROSS_CONF
> +	$(INSTALL) -D -m 0644 package/python-meson/cross-compilation.conf.in \
> +		$(HOST_DIR)/etc/meson/cross-compilation.conf
> +	$(SED) 's;@TARGET_CROSS@;$(TARGET_CROSS);g' \
> +		-e 's;@TARGET_ARCH@;$(HOST_PYTHON_MESON_TARGET_ARCH);g' \
> +		-e 's;@TARGET_ENDIAN@;$(HOST_PYTHON_MESON_TARGET_ENDIAN);g' \
> +		-e 's;@HOST_DIR@;$(HOST_DIR);g' \
> +		$(HOST_DIR)/etc/meson/cross-compilation.conf
> +endef
> +
> +define HOST_PYTHON_MESON_INSTALL_EXE_WRAPPER
> +	$(INSTALL) -D -m 0755 package/python-meson/meson-exe-wrapper.in \
> +		$(HOST_DIR)/usr/bin/meson-exe-wrapper
> +	$(SED) 's;@TARGET_ARCH@;$(HOST_PYTHON_MESON_TARGET_ARCH);g' \
> +		-e 's;@HOST_DIR@;$(HOST_DIR);g' \
> +		-e 's;@TARGET_DIR@;$(TARGET_DIR);g' \
> +		$(HOST_DIR)/usr/bin/meson-exe-wrapper
> +endef
> +
> +HOST_PYTHON_MESON_POST_INSTALL_HOOKS += \
> +	HOST_PYTHON_MESON_REMOVE_GUI_TOOL \
> +	HOST_PYTHON_MESON_INSTALL_CROSS_CONF \
> +	HOST_PYTHON_MESON_INSTALL_EXE_WRAPPER
> +
> +
> +$(eval $(python-package))
> +$(eval $(host-python-package))

Other than that, looks good!

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2] python-meson: new package

[Eric Le Bihan]

Hi!

Le Wed, 8 Jun 2016 22:22:50 +0200,
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> a ?crit :

> > To cross-compile a Meson-based project for the target, its package
> > should:
> > 
> >  - depend on host-python-meson
> >  - invoke the host variant of Meson with the
> >    *--cross-file=$(HOST_DIR)/etc/meson/cross-compilation.conf*
> > option.
> >  - invoke host variant of Ninja to perform the actual build.  
> 
> It would probably be good to write this somewhere, as it is quite
> important to know for users of this build system. But I'm not sure
> where: you're going to have only a host package, so no Config.in file,
> and therefore no help text. And I'm not sure where this could be added
> in the Buildroot manual.

It is true that all the "docs/manual/adding-packages*.txt" files refer
to a dedicated infrastructure, whereas what is needed here is an
example of Makefile using the generic-package infrastructure. There is
an exception to the rule, though: docs/manual/adding-packages-gettext.txt.

So, IMHO, a new document named "docs/manual/adding-packages-meson.txt" should 
fit.

However, I could provide a real package infrastructure named "meson-package",
but as stated in the discussion about adding support for Cargo [1] (the Rust
package manager), to provide such infrastructure, at least one package using
it should also be provided (it is sensible to have a working example of 
the infrastructure).
 
> >  package/Config.in                              |  1 +
> >  package/python-meson/Config.in                 |  9 +++++  
> 
> As discussed, if it's a build system, please add only a host package.

OK. I'll do the same for Ninja.

> > +
> > +define HOST_PYTHON_MESON_REMOVE_GUI_TOOL
> > +	rm -f $(HOST_DIR)/usr/bin/mesongui.py
> > +endef  
> 
> Not sure removing stuff from the host variant is really useful.

This program needs PyQt5, which may not be installed by default by the
most popular GNU/Linux distributions. The user may be tempted to use
it: the execution will fail and this may result in an unnecessary
Buildroot bug report. To avoid this, I chose to remove it.

> Other than that, looks good!

Thanks for the review.

[1] http://lists.busybox.net/pipermail/buildroot/2016-April/158333.html

Regards,

-- 
ELB

[Buildroot] [PATCH 2/2] python-meson: new package

[Thomas Petazzoni]

Hello Eric,

On Thu, 9 Jun 2016 19:59:54 +0200, Eric Le Bihan wrote:

> It is true that all the "docs/manual/adding-packages*.txt" files refer
> to a dedicated infrastructure, whereas what is needed here is an
> example of Makefile using the generic-package infrastructure. There is
> an exception to the rule, though: docs/manual/adding-packages-gettext.txt.
> 
> So, IMHO, a new document named "docs/manual/adding-packages-meson.txt" should 
> fit.

Sounds good.

> However, I could provide a real package infrastructure named "meson-package",
> but as stated in the discussion about adding support for Cargo [1] (the Rust
> package manager), to provide such infrastructure, at least one package using
> it should also be provided (it is sensible to have a working example of 
> the infrastructure).

I would not say one, but at least 4-5 packages, with a pretty good
confidence that more packages would be added later on.

> > > +
> > > +define HOST_PYTHON_MESON_REMOVE_GUI_TOOL
> > > +	rm -f $(HOST_DIR)/usr/bin/mesongui.py
> > > +endef    
> > 
> > Not sure removing stuff from the host variant is really useful.  
> 
> This program needs PyQt5, which may not be installed by default by the
> most popular GNU/Linux distributions. The user may be tempted to use
> it: the execution will fail and this may result in an unnecessary
> Buildroot bug report. To avoid this, I chose to remove it.

We don't bother doing such cleanup steps for host packages, I really
don't think it's worth it.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com