[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Adam Duskett]

Currently, packages that rely on host-python, and are compatible with both
python2 and 3, only build python3 if python3 is selected for the target.

This causes a problem in the following scenario:

- A user does not have a target python selected.
- Package A depends on host-python3
- Package B is compatible with host-python and host-python3

In this scenario, host-python is not needed, as package A and B are
both compatible with python3. However; both host-python and host-python3 will
be built because no target-python has been selected.

To fix this, the following patch series introduces a few minimal changes
that will allow the user to manually select a host version of python.
(Note: I have changed only the packages that I personally use and have tested.)

Adam Duskett (8):
  python: add host-entry
  python3: add host-entry
  python-setuptools: check host-python version
  pkg-waf.mk: check host-python version
  ninja: check host-python version
  libselinux: check host-python version
  setools: check host-python version
  libselinux: check host-python version

 package/Config.in.host                         |  2 ++
 package/libselinux/libselinux.mk               |  2 +-
 package/libsemanage/libsemanage.mk             |  2 +-
 package/ninja/ninja.mk                         |  2 +-
 package/pkg-waf.mk                             | 20 +++++++++++++-------
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 package/python/Config.in.host                  |  6 ++++++
 package/python3/Config.in.host                 |  6 ++++++
 package/setools/setools.mk                     |  6 ++++++
 9 files changed, 45 insertions(+), 11 deletions(-)
 create mode 100644 package/python/Config.in.host
 create mode 100644 package/python3/Config.in.host

-- 
2.14.3

[Buildroot] [PATCH 1/8] python: add selectable host entry

[Adam Duskett]

Add a user selectable entry to build python2 as a host package.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/Config.in.host        | 1 +
 package/python/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index dd6415bba5..7d908c409c 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -41,6 +41,7 @@ menu "Host utilities"
 	source "package/pkgconf/Config.in.host"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
+	source "package/python/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
diff --git a/package/python/Config.in.host b/package/python/Config.in.host
new file mode 100644
index 0000000000..0a8d6d179f
--- /dev/null
+++ b/package/python/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON
+	bool "host python"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
-- 
2.14.3

[Buildroot] [PATCH 2/8] python3: add selectable host entry

[Adam Duskett]

Add a user selectable entry to build python3 as a host package.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/Config.in.host         | 1 +
 package/python3/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python3/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 7d908c409c..19d45e9d9e 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -42,6 +42,7 @@ menu "Host utilities"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
 	source "package/python/Config.in.host"
+	source "package/python3/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
diff --git a/package/python3/Config.in.host b/package/python3/Config.in.host
new file mode 100644
index 0000000000..876661e71b
--- /dev/null
+++ b/package/python3/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON3
+	bool "host python3"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
-- 
2.14.3

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Adam Duskett]

python-setuptools is compatible with both python2 and python3, as such,
there is no need to force python2 as a dependency.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/package/python-setuptools/python-setuptools.mk b/package/python-setuptools/python-setuptools.mk
index 28d4f29b5b..a21494a288 100644
--- a/package/python-setuptools/python-setuptools.mk
+++ b/package/python-setuptools/python-setuptools.mk
@@ -10,10 +10,18 @@ PYTHON_SETUPTOOLS_LICENSE = MIT
 PYTHON_SETUPTOOLS_LICENSE_FILES = LICENSE
 PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif
+
 # recent setuptools versions require bootstrap.py to be invoked
 # before the standard setup process.
 define PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-	cd  $(@D) && $(HOST_DIR)/bin/python ./bootstrap.py
+	cd  $(@D) && $(HOST_DIR)/bin/$(HOST_PYTHON_SETUPTOOLS_PYLIBVER) ./bootstrap.py
 endef
 
 PYTHON_SETUPTOOLS_PRE_CONFIGURE_HOOKS = PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-- 
2.14.3

[Buildroot] [PATCH 4/8] pkg-waf.mk: check host-python version

[Adam Duskett]

waf is compatible with both python2 and python3. Now that there are host
entries for python, the make logic can be changed to check if host-python3
has been selected, and if so, depend on it instead of explicitly depending
only on python2.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/pkg-waf.mk | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/package/pkg-waf.mk b/package/pkg-waf.mk
index e5b606f063..8df287074b 100644
--- a/package/pkg-waf.mk
+++ b/package/pkg-waf.mk
@@ -36,9 +36,6 @@
 
 define inner-waf-package
 
-# We need host-python to run waf
-$(2)_DEPENDENCIES += host-python
-
 $(2)_NEEDS_EXTERNAL_WAF ?= NO
 
 # If the package does not have its own waf, use our own.
@@ -49,6 +46,15 @@ else
 $(2)_WAF = ./waf
 endif
 
+# We need a host-python to run waf
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+$(2)_DEPENDENCIES += host-python3
+$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python3
+else
+$(2)_DEPENDENCIES += host-python
+$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python2
+endif
+
 $(2)_BUILD_OPTS				?=
 $(2)_INSTALL_STAGING_OPTS		?=
 $(2)_INSTALL_TARGET_OPTS		?=
@@ -63,7 +69,7 @@ define $(2)_CONFIGURE_CMDS
 	cd $$(@D) && \
 	$$(TARGET_CONFIGURE_OPTS) \
 	$$($(2)_CONF_ENV) \
-	$$(HOST_DIR)/bin/python2 $$($(2)_WAF) configure \
+	$$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) configure \
 		--prefix=/usr \
 		--libdir=/usr/lib \
 		$$($(2)_CONF_OPTS) \
@@ -78,7 +84,7 @@ endif
 ifndef $(2)_BUILD_CMDS
 define $(2)_BUILD_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		build -j $$(PARALLEL_JOBS) $$($(2)_BUILD_OPTS) \
 		$$($(2)_WAF_OPTS)
 endef
@@ -91,7 +97,7 @@ endif
 ifndef $(2)_INSTALL_STAGING_CMDS
 define $(2)_INSTALL_STAGING_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		install --destdir=$$(STAGING_DIR) \
 		$$($(2)_INSTALL_STAGING_OPTS) \
 		$$($(2)_WAF_OPTS)
@@ -105,7 +111,7 @@ endif
 ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		install --destdir=$$(TARGET_DIR) \
 		$$($(2)_INSTALL_TARGET_OPTS) \
 		$$($(2)_WAF_OPTS)
-- 
2.14.3

[Buildroot] [PATCH 5/8] ninja: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/ninja/ninja.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/ninja/ninja.mk b/package/ninja/ninja.mk
index 259a87d216..b58d1a0f43 100644
--- a/package/ninja/ninja.mk
+++ b/package/ninja/ninja.mk
@@ -9,7 +9,7 @@ NINJA_SITE = $(call github,ninja-build,ninja,$(NINJA_VERSION))
 NINJA_LICENSE = Apache-2.0
 NINJA_LICENSE_FILES = COPYING
 
-HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
+HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_HOST_PYTHON3),host-python3,host-python)
 
 define HOST_NINJA_BUILD_CMDS
 	(cd $(@D); ./configure.py --bootstrap)
-- 
2.14.3

[Buildroot] [PATCH 6/8] libselinux: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/libselinux/libselinux.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/libselinux/libselinux.mk b/package/libselinux/libselinux.mk
index 8ac8000de5..f8e8ea1285 100644
--- a/package/libselinux/libselinux.mk
+++ b/package/libselinux/libselinux.mk
@@ -75,7 +75,7 @@ endef
 HOST_LIBSELINUX_DEPENDENCIES = \
 	host-libsepol host-pcre host-swig
 
-ifeq ($(BR2_PACKAGE_PYTHON3),y)
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
 HOST_LIBSELINUX_DEPENDENCIES += host-python3
 HOST_LIBSELINUX_PYINC = -I$(HOST_DIR)/include/python$(PYTHON3_VERSION_MAJOR)m/
 HOST_LIBSELINUX_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
-- 
2.14.3

[Buildroot] [PATCH 7/8] setools: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/setools/setools.mk | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/package/setools/setools.mk b/package/setools/setools.mk
index 6748c95c23..f4644b091c 100644
--- a/package/setools/setools.mk
+++ b/package/setools/setools.mk
@@ -20,6 +20,12 @@ SETOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
 SETOOLS_DEPENDENCIES += python-enum34
 endif
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_SETOOLS_NEEDS_HOST_PYTHON += python3
+else
+HOST_SETOOLS_NEEDS_HOST_PYTHON += python
+endif
+
 define SETOOLS_FIX_SETUP
 	# By default, setup.py will look for libsepol.a in the host machines
 	# /usr/lib directory. This needs to be changed to the staging directory.
-- 
2.14.3

[Buildroot] [PATCH 8/8] libselinux: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/libsemanage/libsemanage.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/libsemanage/libsemanage.mk b/package/libsemanage/libsemanage.mk
index 829409f527..b9a98c7b6c 100644
--- a/package/libsemanage/libsemanage.mk
+++ b/package/libsemanage/libsemanage.mk
@@ -37,7 +37,7 @@ HOST_LIBSEMANAGE_MAKE_OPTS += \
 	PREFIX=$(HOST_DIR) \
 	SWIG_LIB="$(HOST_DIR)/share/swig/$(SWIG_VERSION)/"
 
-ifeq ($(BR2_PACKAGE_PYTHON3),y)
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
 HOST_LIBSEMANAGE_DEPENDENCIES += host-python3
 HOST_LIBSEMANAGE_MAKE_OPTS += \
 	PYINC="-I$(HOST_DIR)/include/python$(PYTHON3_VERSION_MAJOR)m/" \
-- 
2.14.3

[Buildroot] [PATCH 2/8] python3: add selectable host entry

[Adam Duskett]

Add a user selectable entry to build python3 as a host package.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/Config.in.host         | 1 +
 package/python3/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python3/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 7d908c409c..19d45e9d9e 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -42,6 +42,7 @@ menu "Host utilities"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
 	source "package/python/Config.in.host"
+	source "package/python3/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
diff --git a/package/python3/Config.in.host b/package/python3/Config.in.host
new file mode 100644
index 0000000000..876661e71b
--- /dev/null
+++ b/package/python3/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON3
+	bool "host python3"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
-- 
2.14.3

[Buildroot] [PATCH 4/8] pkg-waf.mk: check host-python version

[Trent Piepho]

On Tue, 2018-01-02 at 10:28 -0500, Adam Duskett wrote:
>  
> +# We need a host-python to run waf
> +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
> +$(2)_DEPENDENCIES += host-python3
> +$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python3
> +else
> +$(2)_DEPENDENCIES += host-python
> +$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python2
> +endif

PYTHON_PATH above is called PYLIBVER in the five existing definitions
of this variable.   PYTHON_PATH already exists, and it the absolute
path in the target dir of sysconfigdata.

So it would be a lot more consistent to name this PYLIBVER.

It also seems like this variable will be re-created in a lot of
packages, if every package that needs to correctly call the host python
must define it.  I wonder if there could be an easier way for python
package files to call the host python correctly.  Like:

$(HOST_PYTHON_EXE)  Program to run for host python, 2 or 3 as selected.
$(HOST_PYTHON2_EXE) If you must have python 2
$(HOST_PYTHON3_EXE) If you must have python 3