Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 0/6] host-python2 hard dependency
@ 2014-03-05 22:04 Samuel Martin
  2014-03-05 22:04 ` [Buildroot] [PATCH 1/6] python: rework python symlinks installation Samuel Martin
                   ` (6 more replies)
  0 siblings, 7 replies; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

Hi all,


Recently, the autobuilders have triggered some host package failures caused by
an unmet host-python(2) dependency.

These failures happen when python3 is selected for the target; in such cases,
the host python is also python3. However, some packages indirectly require a
python2 interpreter available for the host because they have some build scripts
(crda, in a *.py script, uses m2crypto which can only be built for python2) or
use a build-system non-python3-compliant (all packages based on Scons).

This series intends to fix this issue.

First, Buildroot needs to be able to manage the python symlink installation.
The idea behind this is, in the host tree, the python symlink points to the
interpreter the same version as the one in the target tree.
So, we can have:
- if python(2) is installed in the target:
  $(HOST_DIR)/usr/bin/python -> python2
- if python3 is installed in the target:
  $(HOST_DIR)/usr/bin/python -> python3
  $(HOST_DIR)/usr/bin/python2 -> python2.7
This is done in the patches 1/6 and 2/6.

The 3rd patch enhance the python infrastructure to allow setting the host python
requirement for a package using the host-python-package infrastructure. This
patch adds a new variable *_FORCE_HOST_PYTHON to be set to the interpreter name
('python2' or 'python3'), which, if set, will take care of adding the right host
python dependency and using the right interpreter in the build/install commands.

The last 3 packages fix the wrong dependency, so the build failures. Note that,
crda explicitly depends on host-python, while this dependency is automatically
(implicitly) pulled in the python-m2crypto package via the *_FORCE_HOST_PYTHON
usage.


Yours,
Samuel


Samuel Martin (6):
  python: rework python symlinks installation
  python3: rework python symlinks installation
  pkg-python: support host-python dependency different from the python
    in the target
  scons: force host-python dependency to be python2
  python-m2crypto: requires host-python2
  crda: override python interperter

 docs/manual/adding-packages-python.txt             |  8 ++++
 ...file-allow-to-override-python-interpreter.patch | 34 +++++++++++++++
 package/crda/crda.mk                               |  8 +++-
 package/pkg-python.mk                              | 50 ++++++++++++++++++++--
 package/python-m2crypto/python-m2crypto.mk         | 10 +++--
 .../python/python-011-remove-python-symlink.patch  | 29 +++++++++++++
 package/python/python.mk                           | 32 ++++++++++++++
 package/python3/python3.mk                         |  9 ++++
 package/scons/scons.mk                             |  4 +-
 9 files changed, 175 insertions(+), 9 deletions(-)
 create mode 100644 package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch
 create mode 100644 package/python/python-011-remove-python-symlink.patch

--
1.9.0

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 1/6] python: rework python symlinks installation
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
@ 2014-03-05 22:04 ` Samuel Martin
  2014-03-29 11:05   ` Yann E. MORIN
  2014-03-05 22:04 ` [Buildroot] [PATCH 2/6] python3: " Samuel Martin
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

This change adds a patch to python disabling the installation of the
python and python-config symlinks.

This allows Buildroot to control these symlinks' installation:

* the python symlink should be unconditionally installed in the target
  tree, and the python-config symlink in the staging tree, since it is
  only built and installed in the target tree if the user selected it;

* the python and python-config symlinks should only be installed in
  the host tree when python(2) is the selection of the user for the
  target.
  Otherwise, when python3 is selected for the target, the host-python
  may be required to built some packages. In such cases, the python
  symlink should points to python3 (so should the python-config
  symlink) to reflect the staging/target tree.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../python/python-011-remove-python-symlink.patch  | 29 ++++++++++++++++++++
 package/python/python.mk                           | 32 ++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 package/python/python-011-remove-python-symlink.patch

diff --git a/package/python/python-011-remove-python-symlink.patch b/package/python/python-011-remove-python-symlink.patch
new file mode 100644
index 0000000..a9b1daa
--- /dev/null
+++ b/package/python/python-011-remove-python-symlink.patch
@@ -0,0 +1,29 @@
+Remove the python symlink install rules.
+
+The python symlink installation will be handled by Buildroot itself, because
+Buildroot needs to control to what python interpreter (python2 or python3) the
+python symlink points to.
+
+Signed-off-by: Samuel Martin <s.martin49@gmail.com>
+
+diff -purN a/Makefile.pre.in b/Makefile.pre.in
+--- a/Makefile.pre.in	2014-03-02 17:56:56.529132499 +0100
++++ b/Makefile.pre.in	2014-03-02 19:24:02.346491849 +0100
+@@ -857,17 +857,10 @@ $(DESTSHARED):
+ #  $(PYTHON) -> python2 -> python$(VERSION))
+ # Also create equivalent chains for other installed files
+ bininstall:	altbininstall
+-	-if test -f $(DESTDIR)$(BINDIR)/$(PYTHON) -o -h $(DESTDIR)$(BINDIR)/$(PYTHON); \
+-	then rm -f $(DESTDIR)$(BINDIR)/$(PYTHON); \
+-	else true; \
+-	fi
+-	(cd $(DESTDIR)$(BINDIR); $(LN) -s python2$(EXE) $(PYTHON))
+ 	-rm -f $(DESTDIR)$(BINDIR)/python2$(EXE)
+ 	(cd $(DESTDIR)$(BINDIR); $(LN) -s python$(VERSION)$(EXE) python2$(EXE))
+ 	-rm -f $(DESTDIR)$(BINDIR)/python2-config
+ 	(cd $(DESTDIR)$(BINDIR); $(LN) -s python$(VERSION)-config python2-config)
+-	-rm -f $(DESTDIR)$(BINDIR)/python-config
+-	(cd $(DESTDIR)$(BINDIR); $(LN) -s python2-config python-config)
+ 	-test -d $(DESTDIR)$(LIBPC) || $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$(LIBPC)
+ 	-rm -f $(DESTDIR)$(LIBPC)/python2.pc
+ 	(cd $(DESTDIR)$(LIBPC); $(LN) -s python-$(VERSION).pc python2.pc)
diff --git a/package/python/python.mk b/package/python/python.mk
index 216448e..ec9a6d3 100644
--- a/package/python/python.mk
+++ b/package/python/python.mk
@@ -175,8 +175,40 @@ endef
 
 PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_ENSURE_LIBPYTHON_STRIPPED
 
+#
+# Always install the python symlink in the target tree
+#
+define PYTHON_INSTALL_TARGET_PYTHON_SYMLINK
+	ln -sf python2 $(TARGET_DIR)/usr/bin/python
+endef
+
+PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_INSTALL_TARGET_PYTHON_SYMLINK
+
+#
+# Always install the python-config symlink in the staging tree
+#
+define PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK
+	ln -sf python2-config $(STAGING_DIR)/usr/bin/python-config
+endef
+
+PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK
+
 PYTHON_AUTORECONF = YES
 
+#
+# Some packages may have build scripts requiring python2.
+# Only install the python symlink in the host tree if python(2) is enabled
+# for the target.
+#
+ifeq ($(BR2_PACKAGE_PYTHON),y)
+define HOST_PYTHON_INSTALL_PYTHON_SYMLINK
+	ln -sf python2 $(HOST_DIR)/usr/bin/python
+	ln -sf python2-config $(HOST_DIR)/usr/bin/python-config
+endef
+
+HOST_PYTHON_POST_INSTALL_HOOKS += HOST_PYTHON_INSTALL_PYTHON_SYMLINK
+endif
+
 # Provided to other packages
 PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
 
-- 
1.9.0

^ permalink raw reply related	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 2/6] python3: rework python symlinks installation
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
  2014-03-05 22:04 ` [Buildroot] [PATCH 1/6] python: rework python symlinks installation Samuel Martin
@ 2014-03-05 22:04 ` Samuel Martin
  2014-03-29 11:23   ` Yann E. MORIN
  2014-03-05 22:04 ` [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target Samuel Martin
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

This patch reworks the way python3 and python3-config symlink are
installed.

Buildroot wants to control these symlinks' installation:

* the python3 symlink should be unconditionally installed in the target
  tree, and the python3-config symlink in the staging tree, since it is
  the only python package built and installed in the target tree if the
  user selected it;

* the python3 and python3-config symlinks should only be installed in
  the host tree when python3 is the selection of the user for the
  target.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/python3/python3.mk | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index 3d95a6d..19bef1e 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -156,11 +156,20 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
 endif
 
+#
+# Some packages may have build scripts requiring python3, whatever is the
+# python version chosen for the target.
+# Only install the python symlink in the host tree if python3 is enabled
+# for the target.
+#
+ifeq ($(BR2_PACKAGE_PYTHON3),y)
 define HOST_PYTHON3_INSTALL_SYMLINK
 	ln -fs python3 $(HOST_DIR)/usr/bin/python
+	ln -fs python3-config $(HOST_DIR)/usr/bin/python-config
 endef
 
 HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+endif
 
 # Provided to other packages
 PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
-- 
1.9.0

^ permalink raw reply related	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
  2014-03-05 22:04 ` [Buildroot] [PATCH 1/6] python: rework python symlinks installation Samuel Martin
  2014-03-05 22:04 ` [Buildroot] [PATCH 2/6] python3: " Samuel Martin
@ 2014-03-05 22:04 ` Samuel Martin
  2014-03-29 11:47   ` Yann E. MORIN
  2014-03-05 22:04 ` [Buildroot] [PATCH 4/6] scons: force host-python dependency to be python2 Samuel Martin
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

Some packages need a host-python interpreter with a version different
from the one installed in the target to run some build scripts (eg.
scons requires python2 to run, so build any kind of packages even if
the python interpreter selected for the target is python3).

In such cases, we need to add the right host-python dependency to the
package using the host-python-package infrastructure, and we also want
to invoke the right host python interpreter during the build steps.

This patch add a *_FORCE_HOST_PYTHON variable that can be set either
to 'python2' or 'python3'. This variable can be set by any package
using the host-python-package infrastructure can set to force the
python interpreter for the build. This variable also takes care of
setting the right host-python dependency.

This *_FORCE_HOST_PYTHON variable only affects packages using the
host-python-package infrastructure.

If some configure/build/install commands are overloaded in the *.mk
file, the right python interpreter should be explicitly called.

If the package define some tool variable (eg.: SCONS), the variable
should explicitly called the right python interpreter.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/adding-packages-python.txt |  8 ++++++
 package/pkg-python.mk                  | 50 +++++++++++++++++++++++++++++++---
 2 files changed, 54 insertions(+), 4 deletions(-)

diff --git a/docs/manual/adding-packages-python.txt b/docs/manual/adding-packages-python.txt
index b8d5331..da97654 100644
--- a/docs/manual/adding-packages-python.txt
+++ b/docs/manual/adding-packages-python.txt
@@ -144,6 +144,14 @@ therefore only use a few of them, or none.
   setuptools packages) and +HOST_PKG_PYTHON_SETUPTOOLS_INSTALL_OPT+
   (for host setuptools packages).
 
+* +HOST_PYTHON_FOO_FORCE_HOST_PYTHON+, to define the host python
+  interpreter. The usage of this variable is limited to the host
+  packages. The two supported value are +python2+ and +python3+.
+  It will may sure the right host python package is available for
+  the build and will invoke it for the build. If some build steps
+  are overloaded, the right python interpreter must be explicitly
+  called in the commands.
+
 With the Python infrastructure, all the steps required to build and
 install the packages are already defined, and they generally work well
 for most Python-based packages. However, when required, it is still
diff --git a/package/pkg-python.mk b/package/pkg-python.mk
index 512ef66..178b494 100644
--- a/package/pkg-python.mk
+++ b/package/pkg-python.mk
@@ -143,11 +143,29 @@ $(2)_DEPENDENCIES ?= $(filter-out host-python host-python3 host-python-setuptool
 # Target packages need both the python interpreter on the target (for
 # runtime) and the python interpreter on the host (for
 # compilation). However, host packages only need the python
-# interpreter on the host.
+# interpreter on the host, whose version may be enforced by setting
+# the *_FORCE_HOST_PYTHON variable.
+#
+# So:
+# - for target packages, we always depend on the default python interpreter
+#   (the one selected by the config);
+# - for host packages:
+#   - if *_FORCE_HOST_PYTHON is not set, then we depend on use the default
+#     interperter;
+#   - otherwise, we depend on the one requested by *_FORCE_HOST_PYTHON.
+#
 ifeq ($(4),target)
 $(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3 python3,host-python python)
 else
+ifeq ($($(2)_FORCE_HOST_PYTHON),)
 $(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
+else
+ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
+$(2)_DEPENDENCIES += host-python
+else
+$(2)_DEPENDENCIES += host-$$($(2)_FORCE_HOST_PYTHON)
+endif
+endif
 endif
 
 # Setuptools based packages will need host-python-setuptools (both
@@ -161,6 +179,30 @@ $(2)_DEPENDENCIES += host-python-setuptools
 endif
 endif
 
+# Python interpreter to use for building the package.
+#
+# We may want to specify the python interpreter toi be used for building a
+# package, especially for host-packages (target packages must be built using
+# the same version of the interpreter as the one installed on the target).
+#
+# So:
+# - for target packages, we always use the default python interpreter (which
+#   is the same version as the one built and installed on the target);
+# - for host packages:
+#   - if *_FORCE_HOST_PYTHON is not set, then we use use the default
+#     interperter;
+#   - otherwise, we use the one requested by *_FORCE_HOST_PYTHON.
+#
+ifeq ($(4),target)
+$(2)_PYTHON_INTERPRETER = $(HOST_DIR)/usr/bin/python
+else
+ifeq ($($(2)_FORCE_HOST_PYTHON),)
+$(2)_PYTHON_INTERPRETER = $(HOST_DIR)/usr/bin/python
+else
+$(2)_PYTHON_INTERPRETER = $(HOST_DIR)/usr/bin/$($(2)_FORCE_HOST_PYTHON)
+endif
+endif
+
 #
 # Build step. Only define it if not already defined by the package .mk
 # file.
@@ -169,7 +211,7 @@ ifndef $(2)_BUILD_CMDS
 define $(2)_BUILD_CMDS
 	(cd $$($$(PKG)_BUILDDIR)/; \
 		$$($$(PKG)_BASE_ENV) $$($$(PKG)_ENV) \
-		$(HOST_DIR)/usr/bin/python setup.py \
+		$$($(2)_PYTHON_INTERPRETER) setup.py \
 		$$($$(PKG)_BASE_BUILD_TGT) \
 		$$($$(PKG)_BASE_BUILD_OPT) $$($$(PKG)_BUILD_OPT))
 endef
@@ -183,7 +225,7 @@ ifndef $(2)_INSTALL_CMDS
 define $(2)_INSTALL_CMDS
 	(cd $$($$(PKG)_BUILDDIR)/; \
 		$$($$(PKG)_BASE_ENV) $$($$(PKG)_ENV) \
-		$(HOST_DIR)/usr/bin/python setup.py install \
+		$$($(2)_PYTHON_INTERPRETER) setup.py install \
 		$$($$(PKG)_BASE_INSTALL_OPT) $$($$(PKG)_INSTALL_OPT))
 endef
 endif
@@ -196,7 +238,7 @@ ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
 	(cd $$($$(PKG)_BUILDDIR)/; \
 		$$($$(PKG)_BASE_ENV) $$($$(PKG)_ENV) \
-		$(HOST_DIR)/usr/bin/python setup.py install \
+		$$($(2)_PYTHON_INTERPRETER) setup.py install \
 		$$($$(PKG)_BASE_INSTALL_OPT) $$($$(PKG)_INSTALL_OPT))
 endef
 endif
-- 
1.9.0

^ permalink raw reply related	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 4/6] scons: force host-python dependency to be python2
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
                   ` (2 preceding siblings ...)
  2014-03-05 22:04 ` [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target Samuel Martin
@ 2014-03-05 22:04 ` Samuel Martin
  2014-03-29 11:50   ` Yann E. MORIN
  2014-03-05 22:04 ` [Buildroot] [PATCH 5/6] python-m2crypto: requires host-python2 Samuel Martin
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

Scons build-system needs python2 as interpreter (it does not support
python3 yet).

So, we need to force the host dependency to get the python2 interperter
built and available in the host tree to be able to build host-scons
itself and to build scons-based packages, whatever is the python
interpreter for the target.

This patch also makes sure scons will in be called using the right
python interpreter when invoked via $(SCONS).

Fixes:
  http://autobuild.buildroot.net/results/11e6c8c8d79d56ed43daf52e6d6dc80847709926/

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/scons/scons.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/package/scons/scons.mk b/package/scons/scons.mk
index e3fad1b..88396d4 100644
--- a/package/scons/scons.mk
+++ b/package/scons/scons.mk
@@ -10,10 +10,12 @@ SCONS_LICENSE = MIT
 SCONS_LICENSE_FILES = LICENSE.txt
 SCONS_SETUP_TYPE = distutils
 
+HOST_SCONS_FORCE_HOST_PYTHON = python2
+
 HOST_SCONS_INSTALL_OPT = \
 	--install-lib=$(HOST_DIR)/usr/lib/scons-$(SCONS_VERSION)
 
 $(eval $(host-python-package))
 
 # variables used by other packages
-SCONS = $(HOST_DIR)/usr/bin/scons
+SCONS = $(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons
-- 
1.9.0

^ permalink raw reply related	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 5/6] python-m2crypto: requires host-python2
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
                   ` (3 preceding siblings ...)
  2014-03-05 22:04 ` [Buildroot] [PATCH 4/6] scons: force host-python dependency to be python2 Samuel Martin
@ 2014-03-05 22:04 ` Samuel Martin
  2014-03-29 11:51   ` Yann E. MORIN
  2014-03-05 22:04 ` [Buildroot] [PATCH 6/6] crda: override python interperter Samuel Martin
  2014-04-05 15:42 ` [Buildroot] [PATCH 0/6] host-python2 hard dependency Thomas Petazzoni
  6 siblings, 1 reply; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

Acording to its homepage [1], m2crypto does not support python3 yet.

So, this patch forces building the host m2crypto module for pyhton2.

[1] https://pypi.python.org/pypi/M2Crypto

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/python-m2crypto/python-m2crypto.mk | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/package/python-m2crypto/python-m2crypto.mk b/package/python-m2crypto/python-m2crypto.mk
index dd0f77e..62dc720 100644
--- a/package/python-m2crypto/python-m2crypto.mk
+++ b/package/python-m2crypto/python-m2crypto.mk
@@ -10,12 +10,16 @@ PYTHON_M2CRYPTO_SOURCE = M2Crypto-$(PYTHON_M2CRYPTO_VERSION).tar.gz
 PYTHON_M2CRYPTO_SETUP_TYPE = setuptools
 HOST_PYTHON_M2CRYPTO_DEPENDENCIES = host-openssl host-swig
 
-# We need to override the build commands to be able to use build_ext,
-# which accepts the --openssl option.
+# We need to use python2 because m2crypto is not python3 compliant.
+HOST_PYTHON_M2CRYPTO_FORCE_HOST_PYTHON = python2
+
+# * We need to override the build commands to be able to use build_ext,
+#   which accepts the --openssl option.
+# * Use python2 interpreter to avoid trying building some python3 objects.
 define HOST_PYTHON_M2CRYPTO_BUILD_CMDS
 	(cd $(@D); \
 		$(HOST_PKG_PYTHON_SETUPTOOLS_ENV) \
-		$(HOST_DIR)/usr/bin/python setup.py build_ext \
+		$(HOST_DIR)/usr/bin/python2 setup.py build_ext \
 			--openssl=$(HOST_DIR)/usr)
 endef
 
-- 
1.9.0

^ permalink raw reply related	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 6/6] crda: override python interperter
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
                   ` (4 preceding siblings ...)
  2014-03-05 22:04 ` [Buildroot] [PATCH 5/6] python-m2crypto: requires host-python2 Samuel Martin
@ 2014-03-05 22:04 ` Samuel Martin
  2014-03-29 11:56   ` Yann E. MORIN
  2014-04-05 15:42 ` [Buildroot] [PATCH 0/6] host-python2 hard dependency Thomas Petazzoni
  6 siblings, 1 reply; 16+ messages in thread
From: Samuel Martin @ 2014-03-05 22:04 UTC (permalink / raw)
  To: buildroot

During its build, crda calls a python script importing m2crypto module
which requires python2 interpreter.

So, we have a build dependency against host-python.

Also, we make sure this python script is interpreted using the right
python interpreter; we add a patch allowing to enforce this and
explicitly set it.

Fixes:
  http://autobuild.buildroot.net/results/a1e/a1e53d66d3962fd5431e7601937b071a0a3c2084/

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 ...file-allow-to-override-python-interpreter.patch | 34 ++++++++++++++++++++++
 package/crda/crda.mk                               |  8 ++++-
 2 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch

diff --git a/package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch b/package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch
new file mode 100644
index 0000000..2481390
--- /dev/null
+++ b/package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch
@@ -0,0 +1,34 @@
+From 6831024a247fd89c7f7adcbee8a0be991b3caaf4 Mon Sep 17 00:00:00 2001
+From: Samuel Martin <s.martin49@gmail.com>
+Date: Sun, 2 Mar 2014 23:45:44 +0100
+Subject: [PATCH] Makefile: allow to override python interpreter
+
+Signed-off-by: Samuel Martin <s.martin49@gmail.com>
+---
+ Makefile | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/Makefile b/Makefile
+index 1d34bde..d7134ff 100644
+--- a/Makefile
++++ b/Makefile
+@@ -42,6 +42,7 @@ LDLIBS += -lgcrypt
+ reglib.o: keys-gcrypt.c
+ 
+ endif
++PYTHON ?= python2
+ MKDIR ?= mkdir -p
+ INSTALL ?= install
+ 
+@@ -103,7 +104,7 @@ $(REG_BIN):
+ keys-%.c: utils/key2pub.py $(wildcard $(PUBKEY_DIR)/*.pem)
+ 	$(NQ) '  GEN ' $@
+ 	$(NQ) '  Trusted pubkeys:' $(wildcard $(PUBKEY_DIR)/*.pem)
+-	$(Q)./utils/key2pub.py --$* $(wildcard $(PUBKEY_DIR)/*.pem) $@
++	$(Q)$(PYTHON) ./utils/key2pub.py --$* $(wildcard $(PUBKEY_DIR)/*.pem) $@
+ 
+ %.o: %.c regdb.h reglib.h
+ 	$(NQ) '  CC  ' $@
+-- 
+1.9.0
+
diff --git a/package/crda/crda.mk b/package/crda/crda.mk
index ce87d44..0a10816 100644
--- a/package/crda/crda.mk
+++ b/package/crda/crda.mk
@@ -8,7 +8,7 @@ CRDA_VERSION = 1.1.3
 CRDA_SOURCE = crda-$(CRDA_VERSION).tar.bz2
 CRDA_SITE = http://wireless.kernel.org/download/crda
 CRDA_DEPENDENCIES = host-pkgconf host-python-m2crypto \
-	libnl libgcrypt
+	libnl libgcrypt host-python
 CRDA_LICENSE = ISC
 CRDA_LICENSE_FILES = LICENSE
 
@@ -23,10 +23,16 @@ CRDA_NLLIBS += -lnl-3 -lm -lpthread
 CRDA_LDLIBS += `$(STAGING_DIR)/usr/bin/libgcrypt-config --libs`
 endif
 
+# * key2pub.py currently is not python3 compliant (thought python2/pyhton3
+#   compliance could rather easily be achieved.
+# * key2pub.py uses M2Crypto python module, which is only available for
+#   python2, so we have to make sure this script is run suing the python2
+#   interpreter, hence the host-python dependency and the PYTHON var.
 define CRDA_BUILD_CMDS
 	$(TARGET_CONFIGURE_OPTS) \
 		LDLIBS="$(CRDA_LDLIBS)" \
 		NLLIBS="$(CRDA_NLLIBS)" \
+		PYTHON=$(HOST_DIR)/usr/bin/python2 \
 		$(MAKE) all_noverify -C $(@D)
 endef
 
-- 
1.9.0

^ permalink raw reply related	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 1/6] python: rework python symlinks installation
  2014-03-05 22:04 ` [Buildroot] [PATCH 1/6] python: rework python symlinks installation Samuel Martin
@ 2014-03-29 11:05   ` Yann E. MORIN
  0 siblings, 0 replies; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 11:05 UTC (permalink / raw)
  To: buildroot

Samuel, All,

On 2014-03-05 23:04 +0100, Samuel Martin spake thusly:
> This change adds a patch to python disabling the installation of the
> python and python-config symlinks.
[--SNIP--]
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  .../python/python-011-remove-python-symlink.patch  | 29 ++++++++++++++++++++
>  package/python/python.mk                           | 32 ++++++++++++++++++++++
>  2 files changed, 61 insertions(+)
>  create mode 100644 package/python/python-011-remove-python-symlink.patch
> 
> diff --git a/package/python/python-011-remove-python-symlink.patch b/package/python/python-011-remove-python-symlink.patch
> new file mode 100644
> index 0000000..a9b1daa
> --- /dev/null
> +++ b/package/python/python-011-remove-python-symlink.patch
> @@ -0,0 +1,29 @@
> +Remove the python symlink install rules.
> +
> +The python symlink installation will be handled by Buildroot itself, because
> +Buildroot needs to control to what python interpreter (python2 or python3) the
> +python symlink points to.
> +
> +Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> +
> +diff -purN a/Makefile.pre.in b/Makefile.pre.in
> +--- a/Makefile.pre.in	2014-03-02 17:56:56.529132499 +0100
> ++++ b/Makefile.pre.in	2014-03-02 19:24:02.346491849 +0100
> +@@ -857,17 +857,10 @@ $(DESTSHARED):
> + #  $(PYTHON) -> python2 -> python$(VERSION))
> + # Also create equivalent chains for other installed files
> + bininstall:	altbininstall
> +-	-if test -f $(DESTDIR)$(BINDIR)/$(PYTHON) -o -h $(DESTDIR)$(BINDIR)/$(PYTHON); \
> +-	then rm -f $(DESTDIR)$(BINDIR)/$(PYTHON); \
> +-	else true; \
> +-	fi
> +-	(cd $(DESTDIR)$(BINDIR); $(LN) -s python2$(EXE) $(PYTHON))
> + 	-rm -f $(DESTDIR)$(BINDIR)/python2$(EXE)
> + 	(cd $(DESTDIR)$(BINDIR); $(LN) -s python$(VERSION)$(EXE) python2$(EXE))
> + 	-rm -f $(DESTDIR)$(BINDIR)/python2-config
> + 	(cd $(DESTDIR)$(BINDIR); $(LN) -s python$(VERSION)-config python2-config)
> +-	-rm -f $(DESTDIR)$(BINDIR)/python-config
> +-	(cd $(DESTDIR)$(BINDIR); $(LN) -s python2-config python-config)
> + 	-test -d $(DESTDIR)$(LIBPC) || $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$(LIBPC)
> + 	-rm -f $(DESTDIR)$(LIBPC)/python2.pc
> + 	(cd $(DESTDIR)$(LIBPC); $(LN) -s python-$(VERSION).pc python2.pc)

I guess this patch can't go upstream, of course.

But what about adding a configure flag --enable-install-symlinks to
decide whether to install or not to install the symlinks? Such a patch
is probably more involved, of course, but may have a chance of being
accepted upstream.

This comment should not bar this changeset to be applied.

> diff --git a/package/python/python.mk b/package/python/python.mk
> index 216448e..ec9a6d3 100644
> --- a/package/python/python.mk
> +++ b/package/python/python.mk
> @@ -175,8 +175,40 @@ endef
>  
>  PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_ENSURE_LIBPYTHON_STRIPPED
>  
> +#
> +# Always install the python symlink in the target tree
> +#

I know there two other such comments, but the leading and following
empty comment lines are overkill, and most othe comments are not
following this syntax (which I find particularly ugly. Ditto for the
comments below.

> +define PYTHON_INSTALL_TARGET_PYTHON_SYMLINK
> +	ln -sf python2 $(TARGET_DIR)/usr/bin/python
> +endef
> +
> +PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_INSTALL_TARGET_PYTHON_SYMLINK
> +
> +#
> +# Always install the python-config symlink in the staging tree
> +#
> +define PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK
> +	ln -sf python2-config $(STAGING_DIR)/usr/bin/python-config
> +endef
> +
> +PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK
> +
>  PYTHON_AUTORECONF = YES
>  
> +#
> +# Some packages may have build scripts requiring python2.
> +# Only install the python symlink in the host tree if python(2) is enabled

Why 'python(2)' and not simply 'python2'? We know BR2_PACKAGE_PYTHON is
python2. I found this confusing.

> +# for the target.
> +#
> +ifeq ($(BR2_PACKAGE_PYTHON),y)
> +define HOST_PYTHON_INSTALL_PYTHON_SYMLINK
> +	ln -sf python2 $(HOST_DIR)/usr/bin/python
> +	ln -sf python2-config $(HOST_DIR)/usr/bin/python-config
> +endef
> +
> +HOST_PYTHON_POST_INSTALL_HOOKS += HOST_PYTHON_INSTALL_PYTHON_SYMLINK
> +endif
> +
>  # Provided to other packages
>  PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
>  
> -- 
> 1.9.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 2/6] python3: rework python symlinks installation
  2014-03-05 22:04 ` [Buildroot] [PATCH 2/6] python3: " Samuel Martin
@ 2014-03-29 11:23   ` Yann E. MORIN
  0 siblings, 0 replies; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 11:23 UTC (permalink / raw)
  To: buildroot

Samuel, All,

On 2014-03-05 23:04 +0100, Samuel Martin spake thusly:
> This patch reworks the way python3 and python3-config symlink are
> installed.
> 
> Buildroot wants to control these symlinks' installation:
> 
> * the python3 symlink should be unconditionally installed in the target
>   tree, and the python3-config symlink in the staging tree, since it is
>   the only python package built and installed in the target tree if the
>   user selected it;
> 
> * the python3 and python3-config symlinks should only be installed in
>   the host tree when python3 is the selection of the user for the
>   target.
> 
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  package/python3/python3.mk | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/package/python3/python3.mk b/package/python3/python3.mk
> index 3d95a6d..19bef1e 100644
> --- a/package/python3/python3.mk
> +++ b/package/python3/python3.mk
> @@ -156,11 +156,20 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
>  PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
>  endif
>  
> +#
> +# Some packages may have build scripts requiring python3, whatever is the
> +# python version chosen for the target.
> +# Only install the python symlink in the host tree if python3 is enabled
> +# for the target.
> +#

Remove the leading/trailing empty-comment lines.

> +ifeq ($(BR2_PACKAGE_PYTHON3),y)
>  define HOST_PYTHON3_INSTALL_SYMLINK
>  	ln -fs python3 $(HOST_DIR)/usr/bin/python

It took me a while to see how this would not clash with the previous
change: we can only have either python2 *or* python3 on the target, so
we won't have a conflict on the python symlink.

Maybe worth mentionning on the comment, above, and maybe in the commit
log, as well.

If (and I say 'if') one day we are able to install both python2 and
python3 on the target, we'd have to revisit this.

> +	ln -fs python3-config $(HOST_DIR)/usr/bin/python-config
>  endef
>  
>  HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
> +endif
>  
>  # Provided to other packages
>  PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
> -- 
> 1.9.0

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target
  2014-03-05 22:04 ` [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target Samuel Martin
@ 2014-03-29 11:47   ` Yann E. MORIN
  2014-03-29 13:26     ` Thomas Petazzoni
  0 siblings, 1 reply; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 11:47 UTC (permalink / raw)
  To: buildroot

Samuel, All,

On 2014-03-05 23:04 +0100, Samuel Martin spake thusly:
> Some packages need a host-python interpreter with a version different
> from the one installed in the target to run some build scripts (eg.
> scons requires python2 to run, so build any kind of packages even if

s/so/to/

> the python interpreter selected for the target is python3).
> 
> In such cases, we need to add the right host-python dependency to the
> package using the host-python-package infrastructure, and we also want
> to invoke the right host python interpreter during the build steps.
> 
> This patch add a *_FORCE_HOST_PYTHON variable that can be set either

While we are at it: s/add/adds/

Also, I'd name the variable *_NEEDS_HOST_PYTHON (which is more
meaningfull than FORCE.)

> to 'python2' or 'python3'. This variable can be set by any package
> using the host-python-package infrastructure can set to force the
> python interpreter for the build.

This sentence is grammatically incorrect. s/can set//

> This variable also takes care of
> setting the right host-python dependency.
> 
> This *_FORCE_HOST_PYTHON variable only affects packages using the
> host-python-package infrastructure.
> 
> If some configure/build/install commands are overloaded in the *.mk
> file, the right python interpreter should be explicitly called.
> 
> If the package define some tool variable (eg.: SCONS), the variable

s/define/defines/

> should explicitly called the right python interpreter.
> 
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  docs/manual/adding-packages-python.txt |  8 ++++++
>  package/pkg-python.mk                  | 50 +++++++++++++++++++++++++++++++---
>  2 files changed, 54 insertions(+), 4 deletions(-)
> 
> diff --git a/docs/manual/adding-packages-python.txt b/docs/manual/adding-packages-python.txt
> index b8d5331..da97654 100644
> --- a/docs/manual/adding-packages-python.txt
> +++ b/docs/manual/adding-packages-python.txt
> @@ -144,6 +144,14 @@ therefore only use a few of them, or none.
>    setuptools packages) and +HOST_PKG_PYTHON_SETUPTOOLS_INSTALL_OPT+
>    (for host setuptools packages).
>  
> +* +HOST_PYTHON_FOO_FORCE_HOST_PYTHON+, to define the host python
> +  interpreter. The usage of this variable is limited to the host
> +  packages. The two supported value are +python2+ and +python3+.
> +  It will may sure the right host python package is available for

s/may sure/ensure/

> +  the build and will invoke it for the build. If some build steps
> +  are overloaded, the right python interpreter must be explicitly
> +  called in the commands.
> +
>  With the Python infrastructure, all the steps required to build and
>  install the packages are already defined, and they generally work well
>  for most Python-based packages. However, when required, it is still
> diff --git a/package/pkg-python.mk b/package/pkg-python.mk
> index 512ef66..178b494 100644
> --- a/package/pkg-python.mk
> +++ b/package/pkg-python.mk
> @@ -143,11 +143,29 @@ $(2)_DEPENDENCIES ?= $(filter-out host-python host-python3 host-python-setuptool
>  # Target packages need both the python interpreter on the target (for
>  # runtime) and the python interpreter on the host (for
>  # compilation). However, host packages only need the python
> -# interpreter on the host.
> +# interpreter on the host, whose version may be enforced by setting
> +# the *_FORCE_HOST_PYTHON variable.
> +#
> +# So:
> +# - for target packages, we always depend on the default python interpreter
> +#   (the one selected by the config);
> +# - for host packages:
> +#   - if *_FORCE_HOST_PYTHON is not set, then we depend on use the default
> +#     interperter;
> +#   - otherwise, we depend on the one requested by *_FORCE_HOST_PYTHON.
> +#
>  ifeq ($(4),target)
>  $(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3 python3,host-python python)
>  else
> +ifeq ($($(2)_FORCE_HOST_PYTHON),)
>  $(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
> +else
> +ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
> +$(2)_DEPENDENCIES += host-python
> +else
> +$(2)_DEPENDENCIES += host-$$($(2)_FORCE_HOST_PYTHON)

If the user makes a typo in the variable value, we'd end up using
host-TYPO and the build may fail. For example, if the user types
'python' instead of 'python3', we'll end up using python2.

I thionk we should explicitly check for (empty), 'python2' or 'python3',
and abort on any other value:

    ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
    $(2)_DEPENDENCIES += host-python
    else ifeq ($($(2)_FORCE_HOST_PYTHON),python3)
    $(2)_DEPENDENCIES += host-python3
    else
    $$(error Incorrect value '$($(2)_FORCE_HOST_PYTHON)' for $(2)_FORCE_HOST_PYTHON)
    endif
    endif

> +endif
> +endif
>  endif
>  
>  # Setuptools based packages will need host-python-setuptools (both
> @@ -161,6 +179,30 @@ $(2)_DEPENDENCIES += host-python-setuptools
>  endif
>  endif
>  
> +# Python interpreter to use for building the package.
> +#
> +# We may want to specify the python interpreter toi be used for building a
> +# package, especially for host-packages (target packages must be built using
> +# the same version of the interpreter as the one installed on the target).
> +#
> +# So:
> +# - for target packages, we always use the default python interpreter (which
> +#   is the same version as the one built and installed on the target);
> +# - for host packages:
> +#   - if *_FORCE_HOST_PYTHON is not set, then we use use the default
> +#     interperter;
> +#   - otherwise, we use the one requested by *_FORCE_HOST_PYTHON.
> +#
> +ifeq ($(4),target)
> +$(2)_PYTHON_INTERPRETER = $(HOST_DIR)/usr/bin/python
> +else
> +ifeq ($($(2)_FORCE_HOST_PYTHON),)
> +$(2)_PYTHON_INTERPRETER = $(HOST_DIR)/usr/bin/python
> +else
> +$(2)_PYTHON_INTERPRETER = $(HOST_DIR)/usr/bin/$($(2)_FORCE_HOST_PYTHON)

But we do not need to check here, as it would have been done above.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 4/6] scons: force host-python dependency to be python2
  2014-03-05 22:04 ` [Buildroot] [PATCH 4/6] scons: force host-python dependency to be python2 Samuel Martin
@ 2014-03-29 11:50   ` Yann E. MORIN
  0 siblings, 0 replies; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 11:50 UTC (permalink / raw)
  To: buildroot

Samuel, All,

On 2014-03-05 23:04 +0100, Samuel Martin spake thusly:
> Scons build-system needs python2 as interpreter (it does not support
> python3 yet).
> 
> So, we need to force the host dependency to get the python2 interperter
> built and available in the host tree to be able to build host-scons
> itself and to build scons-based packages, whatever is the python
> interpreter for the target.
> 
> This patch also makes sure scons will in be called using the right
> python interpreter when invoked via $(SCONS).
> 
> Fixes:
>   http://autobuild.buildroot.net/results/11e6c8c8d79d56ed43daf52e6d6dc80847709926/
> 
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

If the variable is renamed, my reviewed-by will still be valid.

Regards,
Yann E. MORIN.

> ---
>  package/scons/scons.mk | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/package/scons/scons.mk b/package/scons/scons.mk
> index e3fad1b..88396d4 100644
> --- a/package/scons/scons.mk
> +++ b/package/scons/scons.mk
> @@ -10,10 +10,12 @@ SCONS_LICENSE = MIT
>  SCONS_LICENSE_FILES = LICENSE.txt
>  SCONS_SETUP_TYPE = distutils
>  
> +HOST_SCONS_FORCE_HOST_PYTHON = python2
> +
>  HOST_SCONS_INSTALL_OPT = \
>  	--install-lib=$(HOST_DIR)/usr/lib/scons-$(SCONS_VERSION)
>  
>  $(eval $(host-python-package))
>  
>  # variables used by other packages
> -SCONS = $(HOST_DIR)/usr/bin/scons
> +SCONS = $(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons
> -- 
> 1.9.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 5/6] python-m2crypto: requires host-python2
  2014-03-05 22:04 ` [Buildroot] [PATCH 5/6] python-m2crypto: requires host-python2 Samuel Martin
@ 2014-03-29 11:51   ` Yann E. MORIN
  0 siblings, 0 replies; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 11:51 UTC (permalink / raw)
  To: buildroot

Samuel, All,

On 2014-03-05 23:04 +0100, Samuel Martin spake thusly:
> Acording to its homepage [1], m2crypto does not support python3 yet.
> 
> So, this patch forces building the host m2crypto module for pyhton2.
> 
> [1] https://pypi.python.org/pypi/M2Crypto
> 
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewd-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

If the variable gets renamed, my reviewed-by still stands.

Regards,
Yann E. MORIN.

> ---
>  package/python-m2crypto/python-m2crypto.mk | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/package/python-m2crypto/python-m2crypto.mk b/package/python-m2crypto/python-m2crypto.mk
> index dd0f77e..62dc720 100644
> --- a/package/python-m2crypto/python-m2crypto.mk
> +++ b/package/python-m2crypto/python-m2crypto.mk
> @@ -10,12 +10,16 @@ PYTHON_M2CRYPTO_SOURCE = M2Crypto-$(PYTHON_M2CRYPTO_VERSION).tar.gz
>  PYTHON_M2CRYPTO_SETUP_TYPE = setuptools
>  HOST_PYTHON_M2CRYPTO_DEPENDENCIES = host-openssl host-swig
>  
> -# We need to override the build commands to be able to use build_ext,
> -# which accepts the --openssl option.
> +# We need to use python2 because m2crypto is not python3 compliant.
> +HOST_PYTHON_M2CRYPTO_FORCE_HOST_PYTHON = python2
> +
> +# * We need to override the build commands to be able to use build_ext,
> +#   which accepts the --openssl option.
> +# * Use python2 interpreter to avoid trying building some python3 objects.
>  define HOST_PYTHON_M2CRYPTO_BUILD_CMDS
>  	(cd $(@D); \
>  		$(HOST_PKG_PYTHON_SETUPTOOLS_ENV) \
> -		$(HOST_DIR)/usr/bin/python setup.py build_ext \
> +		$(HOST_DIR)/usr/bin/python2 setup.py build_ext \
>  			--openssl=$(HOST_DIR)/usr)
>  endef
>  
> -- 
> 1.9.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 6/6] crda: override python interperter
  2014-03-05 22:04 ` [Buildroot] [PATCH 6/6] crda: override python interperter Samuel Martin
@ 2014-03-29 11:56   ` Yann E. MORIN
  0 siblings, 0 replies; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 11:56 UTC (permalink / raw)
  To: buildroot

Samuel, All,

On 2014-03-05 23:04 +0100, Samuel Martin spake thusly:
> During its build, crda calls a python script importing m2crypto module
> which requires python2 interpreter.
> 
> So, we have a build dependency against host-python.
> 
> Also, we make sure this python script is interpreted using the right
> python interpreter; we add a patch allowing to enforce this and
> explicitly set it.
> 
> Fixes:
>   http://autobuild.buildroot.net/results/a1e/a1e53d66d3962fd5431e7601937b071a0a3c2084/
> 
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  ...file-allow-to-override-python-interpreter.patch | 34 ++++++++++++++++++++++
>  package/crda/crda.mk                               |  8 ++++-
>  2 files changed, 41 insertions(+), 1 deletion(-)
>  create mode 100644 package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch
> 
> diff --git a/package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch b/package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch
> new file mode 100644
> index 0000000..2481390
> --- /dev/null
> +++ b/package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch
> @@ -0,0 +1,34 @@
> +From 6831024a247fd89c7f7adcbee8a0be991b3caaf4 Mon Sep 17 00:00:00 2001
> +From: Samuel Martin <s.martin49@gmail.com>
> +Date: Sun, 2 Mar 2014 23:45:44 +0100
> +Subject: [PATCH] Makefile: allow to override python interpreter
> +
> +Signed-off-by: Samuel Martin <s.martin49@gmail.com>

Did you push that upstream?

[--SNIP--]
> diff --git a/package/crda/crda.mk b/package/crda/crda.mk
> index ce87d44..0a10816 100644
> --- a/package/crda/crda.mk
> +++ b/package/crda/crda.mk
> @@ -8,7 +8,7 @@ CRDA_VERSION = 1.1.3
>  CRDA_SOURCE = crda-$(CRDA_VERSION).tar.bz2
>  CRDA_SITE = http://wireless.kernel.org/download/crda
>  CRDA_DEPENDENCIES = host-pkgconf host-python-m2crypto \
> -	libnl libgcrypt
> +	libnl libgcrypt host-python
>  CRDA_LICENSE = ISC
>  CRDA_LICENSE_FILES = LICENSE
>  
> @@ -23,10 +23,16 @@ CRDA_NLLIBS += -lnl-3 -lm -lpthread
>  CRDA_LDLIBS += `$(STAGING_DIR)/usr/bin/libgcrypt-config --libs`
>  endif
>  
> +# * key2pub.py currently is not python3 compliant (thought python2/pyhton3

s/thought/though/

> +#   compliance could rather easily be achieved.
> +# * key2pub.py uses M2Crypto python module, which is only available for
> +#   python2, so we have to make sure this script is run suing the python2
> +#   interpreter, hence the host-python dependency and the PYTHON var.
>  define CRDA_BUILD_CMDS
>  	$(TARGET_CONFIGURE_OPTS) \
>  		LDLIBS="$(CRDA_LDLIBS)" \
>  		NLLIBS="$(CRDA_NLLIBS)" \
> +		PYTHON=$(HOST_DIR)/usr/bin/python2 \
>  		$(MAKE) all_noverify -C $(@D)
>  endef

With the nit above fixed:

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target
  2014-03-29 11:47   ` Yann E. MORIN
@ 2014-03-29 13:26     ` Thomas Petazzoni
  2014-03-29 13:58       ` Yann E. MORIN
  0 siblings, 1 reply; 16+ messages in thread
From: Thomas Petazzoni @ 2014-03-29 13:26 UTC (permalink / raw)
  To: buildroot

Dear Yann E. MORIN,

On Sat, 29 Mar 2014 12:47:28 +0100, Yann E. MORIN wrote:

> I thionk we should explicitly check for (empty), 'python2' or 'python3',
> and abort on any other value:
> 
>     ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
>     $(2)_DEPENDENCIES += host-python
>     else ifeq ($($(2)_FORCE_HOST_PYTHON),python3)
>     $(2)_DEPENDENCIES += host-python3
>     else
>     $$(error Incorrect value '$($(2)_FORCE_HOST_PYTHON)' for $(2)_FORCE_HOST_PYTHON)
>     endif
>     endif

Hum, most packages will *not* define <pkg>_FORCE_HOST_PYTHON I believe.
Or do you plan to modify all packages that depend on host-python so
that they all define a value for <pkg>_FORCE_HOST_PYTHON ?

I believe the idea was that if <pkg>_FORCE_HOST_PYTHON is not defined,
then we default to whatever Python version has been selected.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target
  2014-03-29 13:26     ` Thomas Petazzoni
@ 2014-03-29 13:58       ` Yann E. MORIN
  0 siblings, 0 replies; 16+ messages in thread
From: Yann E. MORIN @ 2014-03-29 13:58 UTC (permalink / raw)
  To: buildroot

Thomas, Samuel, All,

On 2014-03-29 14:26 +0100, Thomas Petazzoni spake thusly:
> On Sat, 29 Mar 2014 12:47:28 +0100, Yann E. MORIN wrote:
> 
> > I thionk we should explicitly check for (empty), 'python2' or 'python3',
> > and abort on any other value:
> > 
> >     ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
> >     $(2)_DEPENDENCIES += host-python
> >     else ifeq ($($(2)_FORCE_HOST_PYTHON),python3)
> >     $(2)_DEPENDENCIES += host-python3
> >     else
> >     $$(error Incorrect value '$($(2)_FORCE_HOST_PYTHON)' for $(2)_FORCE_HOST_PYTHON)
> >     endif
> >     endif
> 
> Hum, most packages will *not* define <pkg>_FORCE_HOST_PYTHON I believe.
> Or do you plan to modify all packages that depend on host-python so
> that they all define a value for <pkg>_FORCE_HOST_PYTHON ?
> 
> I believe the idea was that if <pkg>_FORCE_HOST_PYTHON is not defined,
> then we default to whatever Python version has been selected.

That's what is done a few lines above in Samuel's patch:

    +ifeq ($($(2)_FORCE_HOST_PYTHON),)
     $(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
    +else
     [my code proposal here]
    +endif

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [Buildroot] [PATCH 0/6] host-python2 hard dependency
  2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
                   ` (5 preceding siblings ...)
  2014-03-05 22:04 ` [Buildroot] [PATCH 6/6] crda: override python interperter Samuel Martin
@ 2014-04-05 15:42 ` Thomas Petazzoni
  6 siblings, 0 replies; 16+ messages in thread
From: Thomas Petazzoni @ 2014-04-05 15:42 UTC (permalink / raw)
  To: buildroot

Dear Samuel Martin,

On Wed,  5 Mar 2014 23:04:39 +0100, Samuel Martin wrote:

> Samuel Martin (6):
>   python: rework python symlinks installation
>   python3: rework python symlinks installation
>   pkg-python: support host-python dependency different from the python
>     in the target
>   scons: force host-python dependency to be python2
>   python-m2crypto: requires host-python2
>   crda: override python interperter

I've applied your patch series, after making a number of changes,
according to Yann's review. The main change was the renaming of the
variable from <pkg>_FORCE_HOST_PYTHON to <pkg>_NEEDS_HOST_PYTHON.
You'll find all the details in the commit logs of the different commits.

I've also added a separate commit to fix an unrelated issue I stumbled
upon while testing your patch series.

With your patch series, I've been able to test the following
configurations:

 * Build a system with python2 as the target interpreter, with all
   python external modules enabled, the crda package (which needs the
   host python2 interpreter) as well as gpsd and exfat (which need
   host-scons, which in turns needs the host python2 interpreter)

 * Build a system with python3 as the target interpreter, with all
   python external modules enabled, the crda package (which needs the
   host python2 interpreter) as well as gpsd and exfat (which need
   host-scons, which in turns needs the host python2 interpreter)

Both builds went fine.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 16+ messages in thread
end of thread, other threads:[~2014-04-05 15:42 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-05 22:04 [Buildroot] [PATCH 0/6] host-python2 hard dependency Samuel Martin
2014-03-05 22:04 ` [Buildroot] [PATCH 1/6] python: rework python symlinks installation Samuel Martin
2014-03-29 11:05   ` Yann E. MORIN
2014-03-05 22:04 ` [Buildroot] [PATCH 2/6] python3: " Samuel Martin
2014-03-29 11:23   ` Yann E. MORIN
2014-03-05 22:04 ` [Buildroot] [PATCH 3/6] pkg-python: support host-python dependency different from the python in the target Samuel Martin
2014-03-29 11:47   ` Yann E. MORIN
2014-03-29 13:26     ` Thomas Petazzoni
2014-03-29 13:58       ` Yann E. MORIN
2014-03-05 22:04 ` [Buildroot] [PATCH 4/6] scons: force host-python dependency to be python2 Samuel Martin
2014-03-29 11:50   ` Yann E. MORIN
2014-03-05 22:04 ` [Buildroot] [PATCH 5/6] python-m2crypto: requires host-python2 Samuel Martin
2014-03-29 11:51   ` Yann E. MORIN
2014-03-05 22:04 ` [Buildroot] [PATCH 6/6] crda: override python interperter Samuel Martin
2014-03-29 11:56   ` Yann E. MORIN
2014-04-05 15:42 ` [Buildroot] [PATCH 0/6] host-python2 hard dependency Thomas Petazzoni

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox