* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8
@ 2019-10-23 21:19 Grzegorz Blach
2019-10-24 8:05 ` Thomas Petazzoni
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Grzegorz Blach @ 2019-10-23 21:19 UTC (permalink / raw)
To: buildroot
test/import-repeatedly.c uses an embedded python interpreter
and PYTHON_EXTRA_LIBS is used only for building this test case,
so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
to build this test case and don't link the module with libpython3.so.
Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
---
Changes v1 -> v2:
- Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
instead of appending --embed to PYTHON_LIBS
---
package/dbus-python/dbus-python.mk | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/package/dbus-python/dbus-python.mk b/package/dbus-python/dbus-python.mk
index eaf4e5f483..d48208f1d1 100644
--- a/package/dbus-python/dbus-python.mk
+++ b/package/dbus-python/dbus-python.mk
@@ -35,14 +35,16 @@ DBUS_PYTHON_DEPENDENCIES += python3 host-python3
DBUS_PYTHON_CONF_ENV += \
PYTHON=$(HOST_DIR)/bin/python3 \
PYTHON_INCLUDES="`$(STAGING_DIR)/usr/bin/python3-config --includes`" \
- PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`"
+ PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`" \
+ PYTHON_EXTRA_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --libs --embed`"
HOST_DBUS_PYTHON_DEPENDENCIES += host-python3
HOST_DBUS_PYTHON_CONF_ENV += \
PYTHON=$(HOST_DIR)/bin/python3 \
PYTHON_INCLUDES="`$(HOST_DIR)/usr/bin/python3-config --includes`" \
- PYTHON_LIBS="`$(HOST_DIR)/usr/bin/python3-config --ldflags`"
+ PYTHON_LIBS="`$(HOST_DIR)/usr/bin/python3-config --ldflags`" \
+ PYTHON_EXTRA_LIBS="`$(HOST_DIR)/usr/bin/python3-config --libs --embed`"
endif
$(eval $(autotools-package))
--
2.23.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach @ 2019-10-24 8:05 ` Thomas Petazzoni 2019-10-24 8:06 ` Thomas Petazzoni 2019-10-24 17:27 ` [Buildroot] [PATCH v3] " Grzegorz Blach 2019-10-25 16:22 ` [Buildroot] [PATCH] " Thomas Petazzoni 2 siblings, 1 reply; 6+ messages in thread From: Thomas Petazzoni @ 2019-10-24 8:05 UTC (permalink / raw) To: buildroot On Wed, 23 Oct 2019 21:19:21 +0000 Grzegorz Blach <grzegorz@blach.pl> wrote: > test/import-repeatedly.c uses an embedded python interpreter > and PYTHON_EXTRA_LIBS is used only for building this test case, > so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` > to build this test case and don't link the module with libpython3.so. > > Fixes: > - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/ > - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/ > and many similar failures. > > Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> > --- > Changes v1 -> v2: > - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` > instead of appending --embed to PYTHON_LIBS This doesn't change the fix, which I think still isn't correct (but I haven't checked in details). A Python extension should not need the --embed option. Thomas -- Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 2019-10-24 8:05 ` Thomas Petazzoni @ 2019-10-24 8:06 ` Thomas Petazzoni 2019-10-24 8:43 ` Arnout Vandecappelle 0 siblings, 1 reply; 6+ messages in thread From: Thomas Petazzoni @ 2019-10-24 8:06 UTC (permalink / raw) To: buildroot On Thu, 24 Oct 2019 10:05:34 +0200 Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > On Wed, 23 Oct 2019 21:19:21 +0000 > Grzegorz Blach <grzegorz@blach.pl> wrote: > > > test/import-repeatedly.c uses an embedded python interpreter > > and PYTHON_EXTRA_LIBS is used only for building this test case, > > so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` > > to build this test case and don't link the module with libpython3.so. > > > > Fixes: > > - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/ > > - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/ > > and many similar failures. > > > > Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> > > --- > > Changes v1 -> v2: > > - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` > > instead of appending --embed to PYTHON_LIBS > > This doesn't change the fix, which I think still isn't correct (but I > haven't checked in details). A Python extension should not need the > --embed option. And I replied too fast, without reading your commit log. Reading it, it makes more sense. However, I think it should be fixed in the dbus-python build system itself, so that the fix can be upstreamed. Could you have a look at that ? Of course, it's not trivial, because you need to pass --embed for Python >= 3.8, but not for older versions of Python. Thomas -- Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 2019-10-24 8:06 ` Thomas Petazzoni @ 2019-10-24 8:43 ` Arnout Vandecappelle 0 siblings, 0 replies; 6+ messages in thread From: Arnout Vandecappelle @ 2019-10-24 8:43 UTC (permalink / raw) To: buildroot On 24/10/2019 10:06, Thomas Petazzoni wrote: > On Thu, 24 Oct 2019 10:05:34 +0200 > Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > >> On Wed, 23 Oct 2019 21:19:21 +0000 >> Grzegorz Blach <grzegorz@blach.pl> wrote: >> >>> test/import-repeatedly.c uses an embedded python interpreter >>> and PYTHON_EXTRA_LIBS is used only for building this test case, >>> so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` >>> to build this test case and don't link the module with libpython3.so. >>> >>> Fixes: >>> - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/ >>> - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/ >>> and many similar failures. >>> >>> Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> >>> --- >>> Changes v1 -> v2: >>> - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` >>> instead of appending --embed to PYTHON_LIBS >> >> This doesn't change the fix, which I think still isn't correct (but I >> haven't checked in details). A Python extension should not need the >> --embed option. > > And I replied too fast, without reading your commit log. Reading it, it > makes more sense. However, I think it should be fixed in the > dbus-python build system itself, so that the fix can be upstreamed. > Could you have a look at that ? > > Of course, it's not trivial, because you need to pass --embed for > Python >= 3.8, but not for older versions of Python. Actually, it looks like this should be retrieved from sysconfig. So maybe it's just a matter of setting the proper environment variables like we do in pkg-python? In other words, I think the entire logic in dbus-python.mk could be replaced with: DBUS_PYTHON_CONF_ENV = $(PKG_PYTHON_SETUPTOOLS_ENV) HOST_DBUS_PYTHON_CONF_ENV = $(HOST_PKG_PYTHON_SETUPTOOLS_ENV) We may still need to set PYTHON=$(HOST_DIR)/bin/python[23] as well. Regards, Arnout ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH v3] package/dbus-python: Fix build with python 3.8 2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach 2019-10-24 8:05 ` Thomas Petazzoni @ 2019-10-24 17:27 ` Grzegorz Blach 2019-10-25 16:22 ` [Buildroot] [PATCH] " Thomas Petazzoni 2 siblings, 0 replies; 6+ messages in thread From: Grzegorz Blach @ 2019-10-24 17:27 UTC (permalink / raw) To: buildroot Python 3.8 doesn't allow python modules to be linked with libpythonX.Y.so, but applications that embed a python interpreter still need to be linked with libpythonX.Y.so. As a solution -lpythonX.Y need to be moved from PYTHON_LIBS to PYTHON_EXTRA_LIBS, but only for python 3.8 or newer. Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> --- Changes v2 -> v3: - Patch m4/ax_python_devel.m4 instead of setting PYTHON_EXTRA_LIBS Changes v1 -> v2: - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` instead of appending --embed to PYTHON_LIBS Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> --- .../0001-Fix-build-with-python-3.8.patch | 95 +++++++++++++++++++ package/dbus-python/dbus-python.mk | 4 + 2 files changed, 99 insertions(+) create mode 100644 package/dbus-python/0001-Fix-build-with-python-3.8.patch diff --git a/package/dbus-python/0001-Fix-build-with-python-3.8.patch b/package/dbus-python/0001-Fix-build-with-python-3.8.patch new file mode 100644 index 0000000000..db4a0d0b10 --- /dev/null +++ b/package/dbus-python/0001-Fix-build-with-python-3.8.patch @@ -0,0 +1,95 @@ +From 1d7695681f1db3f6e3b487abbffc9665f1422bf4 Mon Sep 17 00:00:00 2001 +From: Grzegorz Blach <grzegorz@blach.pl> +Date: Thu, 24 Oct 2019 15:42:55 +0000 +Subject: [PATCH] Fix build with python 3.8 + +Python 3.8 doesn't allow python modules to be linked with +libpythonX.Y.so, but applications that embed a python interpreter +still need to be linked with libpythonX.Y.so. As a solution +-lpythonX.Y need to be moved from PYTHON_LIBS to PYTHON_EXTRA_LIBS, +but only for python 3.8 or newer. + +Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> +--- + m4/ax_python_devel.m4 | 36 +++++++++++++++++++++++------------- + 1 file changed, 23 insertions(+), 13 deletions(-) + +diff --git a/m4/ax_python_devel.m4 b/m4/ax_python_devel.m4 +index 44dbd83..0de0c6e 100644 +--- a/m4/ax_python_devel.m4 ++++ b/m4/ax_python_devel.m4 +@@ -40,6 +40,7 @@ + # Copyright (c) 2009 Matteo Settenvini <matteo@member.fsf.org> + # Copyright (c) 2009 Horst Knorr <hk_classes@knoda.org> + # Copyright (c) 2013 Daniel Mullner <muellner@math.stanford.edu> ++# Copyright (c) 2019 Grzegorz Blach <grzegorz@blach.pl> + # + # This program is free software: you can redistribute it and/or modify it + # under the terms of the GNU General Public License as published by the +@@ -67,7 +68,7 @@ + # modified version of the Autoconf Macro, you may extend this special + # exception to the GPL to apply to your modified version as well. + +-#serial 21 ++#serial 21 + python 3.8 fixes + + AU_ALIAS([AC_PYTHON_DEVEL], [AX_PYTHON_DEVEL]) + AC_DEFUN([AX_PYTHON_DEVEL],[ +@@ -198,6 +199,11 @@ EOD` + AC_DEFINE_UNQUOTED([HAVE_PYTHON], ["$ac_python_version"], + [If available, contains the Python version number currently in use.]) + ++ # Check if we should omit -lpython in PYTHON_LIBS ++ ac_omit_python_library=`$PYTHON -c "import sys; \ ++ ver = sys.version.split ()[[0]]; \ ++ print (ver >= '3.8.0')"` ++ + # First, the library directory: + ac_python_libdir=`cat<<EOD | $PYTHON - + +@@ -223,18 +229,22 @@ EOD` + # credits goes to momjian, I think. I'd like to put the right name + # in the credits, if someone can point me in the right direction... ? + # +- if test -n "$ac_python_libdir" -a -n "$ac_python_library" +- then +- # use the official shared library +- ac_python_library=`echo "$ac_python_library" | sed "s/^lib//"` +- PYTHON_LIBS="-L$ac_python_libdir -l$ac_python_library" ++ if test "$ac_omit_python_library" = "True"; then ++ PYTHON_LIBS="-L$ac_python_libdir" + else +- # old way: use libpython from python_configdir +- ac_python_libdir=`$PYTHON -c \ +- "from distutils.sysconfig import get_python_lib as f; \ +- import os; \ +- print (os.path.join(f(plat_specific=1, standard_lib=1), 'config'));"` +- PYTHON_LIBS="-L$ac_python_libdir -lpython$ac_python_version" ++ if test -n "$ac_python_libdir" -a -n "$ac_python_library" ++ then ++ # use the official shared library ++ ac_python_library=`echo "$ac_python_library" | sed "s/^lib//"` ++ PYTHON_LIBS="-L$ac_python_libdir -l$ac_python_library" ++ else ++ # old way: use libpython from python_configdir ++ ac_python_libdir=`$PYTHON -c \ ++ "from distutils.sysconfig import get_python_lib as f; \ ++ import os; \ ++ print (os.path.join(f(plat_specific=1, standard_lib=1), 'config'));"` ++ PYTHON_LIBS="-L$ac_python_libdir -lpython$ac_python_version" ++ fi + fi + + if test -z "PYTHON_LIBS"; then +@@ -265,7 +275,7 @@ EOD` + if test -z "$PYTHON_EXTRA_LIBS"; then + PYTHON_EXTRA_LIBS=`$PYTHON -c "import distutils.sysconfig; \ + conf = distutils.sysconfig.get_config_var; \ +- print (conf('LIBS') + ' ' + conf('SYSLIBS'))"` ++ print ('-lpython' + (conf('LDVERSION') or conf('VERSION')) + ' ' + conf('LIBS') + ' ' + conf('SYSLIBS'))"` + fi + AC_MSG_RESULT([$PYTHON_EXTRA_LIBS]) + AC_SUBST(PYTHON_EXTRA_LIBS) +-- +2.23.0 + diff --git a/package/dbus-python/dbus-python.mk b/package/dbus-python/dbus-python.mk index eaf4e5f483..201c7130ec 100644 --- a/package/dbus-python/dbus-python.mk +++ b/package/dbus-python/dbus-python.mk @@ -15,6 +15,10 @@ DBUS_PYTHON_CONF_OPTS = --disable-html-docs --disable-api-docs HOST_DBUS_PYTHON_DEPENDENCIES = host-dbus-glib HOST_DBUS_PYTHON_CONF_OPTS = --disable-html-docs --disable-api-docs +# We patch m4/ax_python_devel.m4 +DBUS_PYTHON_AUTORECONF = YES +HOST_DBUS_PYTHON_AUTORECONF = YES + ifeq ($(BR2_PACKAGE_PYTHON),y) DBUS_PYTHON_DEPENDENCIES += python host-python -- 2.23.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach 2019-10-24 8:05 ` Thomas Petazzoni 2019-10-24 17:27 ` [Buildroot] [PATCH v3] " Grzegorz Blach @ 2019-10-25 16:22 ` Thomas Petazzoni 2 siblings, 0 replies; 6+ messages in thread From: Thomas Petazzoni @ 2019-10-25 16:22 UTC (permalink / raw) To: buildroot On Wed, 23 Oct 2019 21:19:21 +0000 Grzegorz Blach <grzegorz@blach.pl> wrote: > test/import-repeatedly.c uses an embedded python interpreter > and PYTHON_EXTRA_LIBS is used only for building this test case, > so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed` > to build this test case and don't link the module with libpython3.so. > > Fixes: > - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/ > - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/ > and many similar failures. > > Signed-off-by: Grzegorz Blach <grzegorz@blach.pl> Guess what, in the end, after investigating, I agree that this patch was the right solution! PYTHON_LIBS is used when building Python extensions and PYTHON_EXTRA_LIBS is used when building code that embeds a Python interpreter. So this patch made perfect sense. Of course, it would be *much* better for ax_python_devel.m4 to use python-config, rather than do its convoluted logic, but your patch is simple is easy, so I've applied. Thanks! Thomas -- Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-10-25 16:22 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach 2019-10-24 8:05 ` Thomas Petazzoni 2019-10-24 8:06 ` Thomas Petazzoni 2019-10-24 8:43 ` Arnout Vandecappelle 2019-10-24 17:27 ` [Buildroot] [PATCH v3] " Grzegorz Blach 2019-10-25 16:22 ` [Buildroot] [PATCH] " Thomas Petazzoni
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.