[Buildroot] [PATCH v4 0/5] Python 3, simple, external package support

[Buildroot] [PATCH v4 0/5] Python 3, simple, external package support

[Patrick Gerber]

This patch series refactor python support in buildroot to enable the 
use of external package with both; python2 and python3. The packages
python-bottle and python-serial have been adapted as examples.

To support others external package more work is needed as they are
based on python setuptools. It's no more supported by python3 and
should be replaced by "Distribute". This should be done in another
patch series.

Changes between v3 and v4:

* Remove the python-common.mk file and move the PYTHON variable
  declaration into python.mk and python3.mk (Requested by Maxime 
  Ripard)

Changes between v2 and v3:

* Same new menu organisation as v2 but with less modifications in 
  package/Config.in (Requested by Thomas Petazzoni)

* New directory called python-common instead of python generic.  
  (Requested by Thomas Petazzoni)

* No more symlink from python to python3 in staging (Requested by 
  Samuel Martin)

Patrick Gerber (5):
  Python menu reorganization
  Add python3 config directory simlink
  Add PYTHON variable to ease the support of both version
  python-bottle: Convert package to be used with python2 or python3
  python-serial: Convert package to be used with python2 or python3

 package/Config.in                      |    3 ++-
 package/python-bottle/python-bottle.mk |    6 +++---
 package/python-common/Config.in        |   35 ++++++++++++++++++++++++++++++++
 package/python-serial/Config.in        |    1 -
 package/python-serial/python-serial.mk |    6 +++---
 package/python/Config.in               |   14 -------------
 package/python/python.mk               |    6 ++++++
 package/python3/Config.in              |   14 -------------
 package/python3/python3.mk             |   15 ++++++++++++++
 9 files changed, 64 insertions(+), 36 deletions(-)
 create mode 100644 package/python-common/Config.in

-- 
1.7.9.5

[Buildroot] [PATCH v4 1/5] Python menu reorganization

[Patrick Gerber]

In order to have a better support for python external package make the python
menu clearer. Python2 and Python3 are now mutually exclusive

Signed-off-by: Patrick Gerber <kpa_info@yahoo.fr>
---
 package/Config.in               |    3 ++-
 package/python-common/Config.in |   35 +++++++++++++++++++++++++++++++++++
 package/python/Config.in        |   14 --------------
 package/python3/Config.in       |   14 --------------
 4 files changed, 37 insertions(+), 29 deletions(-)
 create mode 100644 package/python-common/Config.in

diff --git a/package/Config.in b/package/Config.in
index 2fad94d..5f2c1bd 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -315,9 +315,10 @@ endmenu
 endif
 source "package/microperl/Config.in"
 source "package/php/Config.in"
+source "package/python-common/Config.in"
 source "package/python/Config.in"
 source "package/python3/Config.in"
-if BR2_PACKAGE_PYTHON
+if (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3)
 menu "external python modules"
 source "package/python-bottle/Config.in"
 source "package/python-dpkt/Config.in"
diff --git a/package/python-common/Config.in b/package/python-common/Config.in
new file mode 100644
index 0000000..6523c21
--- /dev/null
+++ b/package/python-common/Config.in
@@ -0,0 +1,35 @@
+config BR2_PACKAGE_PYTHON_COMMON
+	bool "Python"
+	depends on BR2_USE_WCHAR
+	# uses fork()
+	depends on BR2_USE_MMU
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
+
+comment "python3 requires a toolchain with WCHAR support"
+	depends on !BR2_USE_WCHAR
+
+choice
+	depends on BR2_PACKAGE_PYTHON_COMMON
+	prompt "Python interpreter"
+	help
+	  Select which version of Python you would like to use.
+
+config BR2_PACKAGE_PYTHON
+	bool "python2"
+	select BR2_PACKAGE_LIBFFI
+	help
+	  Use Python 2.x 
+
+
+config BR2_PACKAGE_PYTHON3
+	bool "python3"
+	# uses fork()
+	depends on BR2_USE_MMU
+	select BR2_PACKAGE_LIBFFI
+	help
+	  Use Python 3.x 
+
+endchoice
diff --git a/package/python/Config.in b/package/python/Config.in
index 4089cce..b4fd6c2 100644
--- a/package/python/Config.in
+++ b/package/python/Config.in
@@ -1,17 +1,3 @@
-config BR2_PACKAGE_PYTHON
-	bool "python"
-	depends on BR2_USE_WCHAR
-	# uses fork()
-	depends on BR2_USE_MMU
-	select BR2_PACKAGE_LIBFFI
-	help
-	  The python language interpreter.
-
-	  http://www.python.org/
-
-comment "python requires a toolchain with WCHAR support"
-	depends on !BR2_USE_WCHAR
-
 if BR2_PACKAGE_PYTHON
 
 choice
diff --git a/package/python3/Config.in b/package/python3/Config.in
index 5959d31..279f31b 100644
--- a/package/python3/Config.in
+++ b/package/python3/Config.in
@@ -1,17 +1,3 @@
-config BR2_PACKAGE_PYTHON3
-	bool "python3"
-	depends on BR2_USE_WCHAR
-	# uses fork()
-	depends on BR2_USE_MMU
-	select BR2_PACKAGE_LIBFFI
-	help
-	  The python language interpreter.
-
-	  http://www.python.org/
-
-comment "python3 requires a toolchain with WCHAR support"
-	depends on !BR2_USE_WCHAR
-
 if BR2_PACKAGE_PYTHON3
 
 choice
-- 
1.7.9.5

[Buildroot] [PATCH v4 2/5] Add python3 config directory simlink

[Patrick Gerber]

When using host python to install an external module, the setup tool
look for some files in python3.3/config/. The python3 package config
directory is called config-3.3m. This is why we need to alias config
to config-3.3m.

Signed-off-by: Patrick Gerber <kpa_info@yahoo.fr>
---
 package/python3/python3.mk |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index 7551a7b..74af9b4 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -160,6 +160,15 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
 endif
 
+# Alias the python3.3/config-3.3m to python3.3/config
+# This is needed when installing an external python module, because
+# the setup is looking for files in python3.3/config/
+define HOST_PYTHON3_INSTALL_SYMLINK
+	ln -fs config-3.3m $(HOST_DIR)/usr/lib/python3.3/config
+endef
+
+HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+
 ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y)
 define PYTHON3_REMOVE_MODULES_FILES
 	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) \
-- 
1.7.9.5

[Buildroot] [PATCH v4 3/5] Add PYTHON variable to ease the support of both version

[Patrick Gerber]

Define a new variable called PYTHON that contains either "python" or
"python3" depending of the selected Python version. This is needed
to easily set dependencies and to call the right version of python
for packages that support both version of python.

Signed-off-by: Patrick Gerber <kpa_info@yahoo.fr>
---
 package/python/python.mk   |    6 ++++++
 package/python3/python3.mk |    6 ++++++
 2 files changed, 12 insertions(+)

diff --git a/package/python/python.mk b/package/python/python.mk
index 71591b7..6a7de43 100644
--- a/package/python/python.mk
+++ b/package/python/python.mk
@@ -53,6 +53,12 @@ HOST_PYTHON_DEPENDENCIES = host-expat host-zlib
 
 PYTHON_INSTALL_STAGING = YES
 
+# The PYTHON variable could be used by external python package to
+# works with both python's version (python and python3)
+ifeq ($(BR2_PACKAGE_PYTHON),y)
+	PYTHON = python
+endif
+
 ifeq ($(BR2_PACKAGE_PYTHON_READLINE),y)
 PYTHON_DEPENDENCIES += readline
 endif
diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index 74af9b4..44136fc 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -55,6 +55,12 @@ HOST_PYTHON3_DEPENDENCIES = host-expat host-zlib
 
 PYTHON3_INSTALL_STAGING = YES
 
+# The PYTHON variable could be used by external python package to
+# works with both python's version (python and python3)
+ifeq ($(BR2_PACKAGE_PYTHON3),y)
+	PYTHON = python3
+endif
+
 ifeq ($(BR2_PACKAGE_PYTHON3_READLINE),y)
 PYTHON3_DEPENDENCIES += readline
 endif
-- 
1.7.9.5

[Buildroot] [PATCH v4 4/5] python-bottle: Convert package to be used with python2 or python3

[Patrick Gerber]

Signed-off-by: Patrick Gerber <kpa_info@yahoo.fr>
---
 package/python-bottle/python-bottle.mk |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/package/python-bottle/python-bottle.mk b/package/python-bottle/python-bottle.mk
index 370b31b..b8a1847 100644
--- a/package/python-bottle/python-bottle.mk
+++ b/package/python-bottle/python-bottle.mk
@@ -7,16 +7,16 @@
 PYTHON_BOTTLE_VERSION = 0.11.4
 PYTHON_BOTTLE_SOURCE = bottle-$(PYTHON_BOTTLE_VERSION).tar.gz
 PYTHON_BOTTLE_SITE = http://pypi.python.org/packages/source/b/bottle
-PYTHON_BOTTLE_DEPENDENCIES = python
+PYTHON_BOTTLE_DEPENDENCIES = $(PYTHON)
 PYTHON_BOTTLE_LICENSE = MIT
 # README.rst refers to the file "LICENSE" but it's not included
 
 define PYTHON_BOTTLE_BUILD_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py build)
 endef
 
 define PYTHON_BOTTLE_INSTALL_TARGET_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py install --prefix=$(TARGET_DIR)/usr)
 endef
 
 $(eval $(generic-package))
-- 
1.7.9.5

[Buildroot] [PATCH v4 5/5] python-serial: Convert package to be used with python2 or python3

[Patrick Gerber]

Signed-off-by: Patrick Gerber <kpa_info@yahoo.fr>
---
 package/python-serial/Config.in        |    1 -
 package/python-serial/python-serial.mk |    6 +++---
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/package/python-serial/Config.in b/package/python-serial/Config.in
index ea090bd..55a9cc1 100644
--- a/package/python-serial/Config.in
+++ b/package/python-serial/Config.in
@@ -1,6 +1,5 @@
 config BR2_PACKAGE_PYTHON_SERIAL
 	bool "python-serial"
-	depends on BR2_PACKAGE_PYTHON
 	help
 	  python-serial is a Python library to access serial ports.
 
diff --git a/package/python-serial/python-serial.mk b/package/python-serial/python-serial.mk
index fb6b31c..6528a6b 100644
--- a/package/python-serial/python-serial.mk
+++ b/package/python-serial/python-serial.mk
@@ -8,14 +8,14 @@ PYTHON_SERIAL_VERSION = 2.6
 PYTHON_SERIAL_SOURCE  = pyserial-$(PYTHON_SERIAL_VERSION).tar.gz
 PYTHON_SERIAL_SITE    = http://pypi.python.org/packages/source/p/pyserial/
 
-PYTHON_SERIAL_DEPENDENCIES = python
+PYTHON_SERIAL_DEPENDENCIES = $(PYTHON)
 
 define PYTHON_SERIAL_BUILD_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py build)
 endef
 
 define PYTHON_SERIAL_INSTALL_TARGET_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py install --prefix=$(TARGET_DIR)/usr)
 endef
 
 $(eval $(generic-package))
-- 
1.7.9.5

[Buildroot] [PATCH v4 0/5] Python 3, simple, external package support

[Thomas Petazzoni]

Dear Patrick Gerber,

On Fri, 15 Feb 2013 11:00:07 +0100, Patrick Gerber wrote:
> This patch series refactor python support in buildroot to enable the 
> use of external package with both; python2 and python3. The packages
> python-bottle and python-serial have been adapted as examples.
> 
> To support others external package more work is needed as they are
> based on python setuptools. It's no more supported by python3 and
> should be replaced by "Distribute". This should be done in another
> patch series.

It's been a very long time (one year), but I have sent a patch series
that bumps Python 2.x and 3.x to their latest version, and adds the
possibility of building Python external modules with Python 3.

If you have some time, I would definitely be interested by receiving
comments, and testing reports. For details, see:

  http://lists.busybox.net/pipermail/buildroot/2014-February/089309.html

The code is available at:

  http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=python3-pkg

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com