[Buildroot] [PATCH 1/4] trace-cmd: fix largefile build problem

[Buildroot] [PATCH 1/4] trace-cmd: fix largefile build problem

[Thomas Petazzoni]

The source files of trace-cmd already define _LARGEFILE64_SOURCE, but
Buildroot passes it again on the build command line through CFLAGS,
which causes a build error. We fix this by filtering out our
definition of _LARGEFILE64_SOURCE from the CFLAGS before passing them
to the trace-cmd Makefile.

Fixes the warning visible at:

 http://autobuild.buildroot.net/results/1c4567a230940a5287d03e2a5c2c7afbdc64cd36/build-end.log

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/trace-cmd/trace-cmd.mk | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/package/trace-cmd/trace-cmd.mk b/package/trace-cmd/trace-cmd.mk
index ada0306..7a750f9 100644
--- a/package/trace-cmd/trace-cmd.mk
+++ b/package/trace-cmd/trace-cmd.mk
@@ -11,9 +11,13 @@ TRACE_CMD_INSTALL_STAGING = YES
 TRACE_CMD_LICENSE = GPLv2 LGPLv2.1
 TRACE_CMD_LICENSE_FILES = COPYING COPYING.LIB
 
+# trace-cmd already defines _LARGEFILE64_SOURCE when necessary,
+# redefining it on the command line causes build problems.
+TRACE_CMD_CFLAGS=$(filter-out -D_LARGEFILE64_SOURCE,$(TARGET_CFLAGS)) -D_GNU_SOURCE
+
 define TRACE_CMD_BUILD_CMDS
 	$(MAKE) $(TARGET_CONFIGURE_OPTS) \
-		CFLAGS="$(TARGET_CFLAGS) -D_GNU_SOURCE" \
+		CFLAGS="$(TRACE_CMD_CFLAGS)" \
 		-C $(@D) all
 endef
 
-- 
1.8.1.2

[Buildroot] [PATCH 2/4] python: fix python-config for cross-compilation

[Thomas Petazzoni]

Some packages (trace-cmd) use python-config to find out which headers
and libraries should be used to link against the Python libraries. By
default, python-config returns paths that are inappropriate for
cross-compilation: this patch fixes this by prepending those paths by
the staging directory location.

We also fix one of the python*-config file to actually be a symbolic
link to the script, rather than a copy of it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/python/python.mk | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/package/python/python.mk b/package/python/python.mk
index df02b35..dfa0686 100644
--- a/package/python/python.mk
+++ b/package/python/python.mk
@@ -134,14 +134,27 @@ PYTHON_MAKE_ENV = \
 	PYTHON_MODULES_INCLUDE=$(STAGING_DIR)/usr/include \
 	PYTHON_MODULES_LIB="$(STAGING_DIR)/lib $(STAGING_DIR)/usr/lib"
 
-# python distutils adds -L$LIBDIR when linking binary extensions, causing
-# trouble for cross compilation
-define PYTHON_FIXUP_LIBDIR
+# 1. python distutils adds -L$LIBDIR when linking binary extensions,
+# causing trouble for cross compilation.
+#
+# 2. adjust the python-config in the staging directory so that it
+# returns include paths that are valid on the build machine (i.e
+# prefixed by the STAGING_DIR).
+#
+# 3. make sure all python-*config are symbolic links to
+# python2.7-config, instead of being another file.
+#
+define PYTHON_FIXUP_DIRS
 	$(SED) 's|^LIBDIR=.*|LIBDIR= $(STAGING_DIR)/usr/lib|' \
 	   $(STAGING_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/config/Makefile
+	$(SED) "s|'-I'|'-I$(STAGING_DIR)'|" \
+	   $(STAGING_DIR)/usr/bin/python$(PYTHON_VERSION_MAJOR)-config
+	$(RM) -f $(STAGING_DIR)/usr/bin/python-config
+	ln -sf $(STAGING_DIR)/usr/bin/python$(PYTHON_VERSION_MAJOR)-config \
+		$(STAGING_DIR)/usr/bin/python-config
 endef
 
-PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_FIXUP_LIBDIR
+PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_FIXUP_DIRS
 
 #
 # Remove useless files. In the config/ directory, only the Makefile
-- 
1.8.1.2

[Buildroot] [PATCH 3/4] python3: fix python-config for cross-compilation

[Thomas Petazzoni]

Some packages (trace-cmd) use python3-config to find out which headers
and libraries should be used to link against the Python libraries. By
default, python3-config returns paths that are inappropriate for
cross-compilation: this patch fixes this by prepending those paths by
the staging directory location.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/python3/python3.mk | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index a8a1942..e6cf8d4 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -124,14 +124,20 @@ PYTHON3_MAKE_ENV = \
 	PYTHON_MODULES_INCLUDE=$(STAGING_DIR)/usr/include \
 	PYTHON_MODULES_LIB="$(STAGING_DIR)/lib $(STAGING_DIR)/usr/lib"
 
-# python distutils adds -L$LIBDIR when linking binary extensions, causing
-# trouble for cross compilation
-define PYTHON3_FIXUP_LIBDIR
+# 1. python distutils adds -L$LIBDIR when linking binary extensions,
+# causing trouble for cross compilation.
+#
+# 2. adjust the python-config in the staging directory so that it
+# returns include paths that are valid on the build machine (i.e
+# prefixed by the STAGING_DIR).
+define PYTHON3_FIXUP_DIRS
 	$(SED) 's|^LIBDIR=.*|LIBDIR= $(STAGING_DIR)/usr/lib|' \
 	   $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-3.3m/Makefile
+	$(SED) "s|'-I'|'-I$(STAGING_DIR)'|" \
+	   $(STAGING_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
 endef
 
-PYTHON3_POST_INSTALL_STAGING_HOOKS += PYTHON3_FIXUP_LIBDIR
+PYTHON3_POST_INSTALL_STAGING_HOOKS += PYTHON3_FIXUP_DIRS
 
 #
 # Remove useless files. In the config/ directory, only the Makefile
-- 
1.8.1.2

[Buildroot] [PATCH 4/4] trace-cmd: improve Python support

[Thomas Petazzoni]

Unless NO_PYTHON is defined, the trace-cmd build system calls the host
python and tries to use it, causing build failures. So we now pass
NO_PYTHON properly when Python is not available for the target.

Fixes:

  http://autobuild.buildroot.net/results/1c4567a230940a5287d03e2a5c2c7afbdc64cd36/build-end.log

We also fix the trace-cmd build to properly link against the target
Python library, when Python is enabled for the target.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../trace-cmd-0001-support-for-python-config.patch | 48 ++++++++++++++++++++++
 package/trace-cmd/trace-cmd.mk                     | 13 ++++++
 2 files changed, 61 insertions(+)
 create mode 100644 package/trace-cmd/trace-cmd-0001-support-for-python-config.patch

diff --git a/package/trace-cmd/trace-cmd-0001-support-for-python-config.patch b/package/trace-cmd/trace-cmd-0001-support-for-python-config.patch
new file mode 100644
index 0000000..dc9d1d2
--- /dev/null
+++ b/package/trace-cmd/trace-cmd-0001-support-for-python-config.patch
@@ -0,0 +1,48 @@
+From 1fb959ec7c2e7f1d663db3125f190ead2f3be91c Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+Date: Mon, 23 Sep 2013 17:55:00 +0200
+Subject: [PATCH] Support for PYTHON_CONFIG
+
+This patch adds support for a PYTHON_CONFIG variable, which can be
+overriden from the make command line. In cross-compilation cases, it
+allows to override which python-config program should be used to find
+the Python headers and libraries.
+
+We also remove the flags added from the distutils LINKFORSHARED
+variable, as they come from the host build, and are anyway already
+provided by the python-config --ldflags option.
+
+Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+---
+ Makefile | 8 ++++----
+ 1 file changed, 4 insertions(+), 4 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index 83329ca..5c5c71b 100644
+--- a/Makefile
++++ b/Makefile
+@@ -81,7 +81,8 @@ PYTHON		:= ctracecmd.so
+ PYTHON_GUI	:= ctracecmd.so ctracecmdgui.so
+ 
+ # Can build python?
+-ifeq ($(shell sh -c "python-config --includes > /dev/null 2>&1 && echo y"), y)
++PYTHON_CONFIG = python-config
++ifeq ($(shell sh -c "$(PYTHON_CONFIG) --includes > /dev/null 2>&1 && echo y"), y)
+ 	PYTHON_PLUGINS := plugin_python.so
+ 	BUILD_PYTHON := $(PYTHON) $(PYTHON_PLUGINS)
+ 	PYTHON_SO_INSTALL := ctracecmd.install
+@@ -546,9 +547,8 @@ clean:
+ 
+ ##### PYTHON STUFF #####
+ 
+-PYTHON_INCLUDES = `python-config --includes`
+-PYTHON_LDFLAGS = `python-config --ldflags` \
+-		$(shell python -c "import distutils.sysconfig; print distutils.sysconfig.get_config_var('LINKFORSHARED')")
++PYTHON_INCLUDES = `$(PYTHON_CONFIG) --includes`
++PYTHON_LDFLAGS = `$(PYTHON_CONFIG) --ldflags`
+ PYGTK_CFLAGS = `pkg-config --cflags pygtk-2.0`
+ 
+ ctracecmd.so: $(TCMD_LIB_OBJS) ctracecmd.i
+-- 
+1.8.1.2
+
diff --git a/package/trace-cmd/trace-cmd.mk b/package/trace-cmd/trace-cmd.mk
index 7a750f9..191eb6d 100644
--- a/package/trace-cmd/trace-cmd.mk
+++ b/package/trace-cmd/trace-cmd.mk
@@ -15,9 +15,22 @@ TRACE_CMD_LICENSE_FILES = COPYING COPYING.LIB
 # redefining it on the command line causes build problems.
 TRACE_CMD_CFLAGS=$(filter-out -D_LARGEFILE64_SOURCE,$(TARGET_CFLAGS)) -D_GNU_SOURCE
 
+ifeq ($(BR2_PACKAGE_PYTHON)$(BR2_PACKAGE_PYTHON3),)
+TRACE_CMD_MAKE_OPTS += NO_PYTHON=1
+else ifeq ($(BR2_PACKAGE_PYTHON),y)
+TRACE_CMD_DEPENDENCIES += python host-swig
+TRACE_CMD_MAKE_OPTS += \
+	PYTHON_CONFIG=$(STAGING_DIR)/usr/bin/python$(PYTHON_VERSION_MAJOR)-config
+else ifeq ($(BR2_PACKAGE_PYTHON3),y)
+TRACE_CMD_DEPENDENCIES += python3 host-swig
+TRACE_CMD_MAKE_OPTS += \
+	PYTHON_CONFIG=$(STAGING_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)-config
+endif
+
 define TRACE_CMD_BUILD_CMDS
 	$(MAKE) $(TARGET_CONFIGURE_OPTS) \
 		CFLAGS="$(TRACE_CMD_CFLAGS)" \
+		$(TRACE_CMD_MAKE_OPTS) \
 		-C $(@D) all
 endef
 
-- 
1.8.1.2

[Buildroot] [PATCH 1/4] trace-cmd: fix largefile build problem

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> The source files of trace-cmd already define _LARGEFILE64_SOURCE, but
 Thomas> Buildroot passes it again on the build command line through CFLAGS,
 Thomas> which causes a build error. We fix this by filtering out our
 Thomas> definition of _LARGEFILE64_SOURCE from the CFLAGS before passing them
 Thomas> to the trace-cmd Makefile.

 Thomas> Fixes the warning visible at:

 Thomas>  http://autobuild.buildroot.net/results/1c4567a230940a5287d03e2a5c2c7afbdc64cd36/build-end.log

It's just a warning, but ok - Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/4] python: fix python-config for cross-compilation

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 Thomas> Some packages (trace-cmd) use python-config to find out which
 Thomas> headers and libraries should be used to link against the Python
 Thomas> libraries. By default, python-config returns paths that are
 Thomas> inappropriate for cross-compilation: this patch fixes this by
 Thomas> prepending those paths by the staging directory location.

My first thought was that this should just use PYTHON_CONFIG_SCRIPTS,
but looking closer at it, I see that it's actually a python script so it
won't work. It would be good to add a comment about this.

The shebang reads:

#!/usr/bin/python2.7

Now that we go to the effort to build a host-python I think it makes
sense to fixup the shebang as well - But:

./output/host/usr/bin/python output/staging/usr/bin/python2.7-config --cflags
-I/home/peko/source/buildroot/output/host/usr/include/python2.7 -I/home/peko/source/buildroot/output/host/usr/include/python2.7 -fno-strict-aliasing -O2 -I/home/peko/source/buildroot/output/host/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

E.G. your s/-I/-I$(STAGING_DIR)/ trick won't work (and it didn't work
before if the host python wasn't built with --prefix=/usr).

 Thomas> We also fix one of the python*-config file to actually be a symbolic
 Thomas> link to the script, rather than a copy of it.

This I don't see here:

ls -lah output/staging/usr/bin/python*-config
-rwxr-xr-x 1 peko peko 1.6K Oct  1 23:50 output/staging/usr/bin/python2.7-config
lrwxrwxrwx 1 peko peko   16 Oct  1 23:50 output/staging/usr/bin/python2-config -> python2.7-config
lrwxrwxrwx 1 peko peko   14 Oct  1 23:50 output/staging/usr/bin/python-config -> python2-config

Care to fix these issues and resubmit?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/4] python: fix python-config for cross-compilation

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Tue, 01 Oct 2013 23:58:32 +0200, Peter Korsgaard wrote:

> My first thought was that this should just use PYTHON_CONFIG_SCRIPTS,
> but looking closer at it, I see that it's actually a python script so it
> won't work. It would be good to add a comment about this.

Ok.

> The shebang reads:
> 
> #!/usr/bin/python2.7
> 
> Now that we go to the effort to build a host-python I think it makes
> sense to fixup the shebang as well - But:

I am not sure. We build host-python only to be able to build the Python
interpreter for the target. For all the rest, we already require a
Python interpreter to be installed on the host machine. The current
situation is a bit confusing: some packages that need a Python
interpreter at build time do depend on host-python, some other packages
do not depend on host-python, and instead rely on the fact that
Buildroot verifies that a Python interpreter is already available on
the build machine.

I believe this is something that should be clarified.

> ./output/host/usr/bin/python output/staging/usr/bin/python2.7-config --cflags
> -I/home/peko/source/buildroot/output/host/usr/include/python2.7 -I/home/peko/source/buildroot/output/host/usr/include/python2.7 -fno-strict-aliasing -O2 -I/home/peko/source/buildroot/output/host/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
> 
> E.G. your s/-I/-I$(STAGING_DIR)/ trick won't work (and it didn't work
> before if the host python wasn't built with --prefix=/usr).

Right, from the above, I am not sure we want to change the shebang of
this script.

>  Thomas> We also fix one of the python*-config file to actually be a symbolic
>  Thomas> link to the script, rather than a copy of it.
> 
> This I don't see here:
> 
> ls -lah output/staging/usr/bin/python*-config
> -rwxr-xr-x 1 peko peko 1.6K Oct  1 23:50 output/staging/usr/bin/python2.7-config
> lrwxrwxrwx 1 peko peko   16 Oct  1 23:50 output/staging/usr/bin/python2-config -> python2.7-config
> lrwxrwxrwx 1 peko peko   14 Oct  1 23:50 output/staging/usr/bin/python-config -> python2-config
> 
> Care to fix these issues and resubmit?

Huh? Odd, I'll have to test this once again then. However, before
resending a new version, I'd like to see how we're going to clarify the
host-python thing vs. Python as a mandatory dependency.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/4] python: fix python-config for cross-compilation

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> The shebang reads:
 >> 
 >> #!/usr/bin/python2.7
 >> 
 >> Now that we go to the effort to build a host-python I think it makes
 >> sense to fixup the shebang as well - But:

 Thomas> I am not sure. We build host-python only to be able to build
 Thomas> the Python interpreter for the target. For all the rest, we
 Thomas> already require a Python interpreter to be installed on the
 Thomas> host machine. The current situation is a bit confusing: some
 Thomas> packages that need a Python interpreter at build time do depend
 Thomas> on host-python, some other packages do not depend on
 Thomas> host-python, and instead rely on the fact that Buildroot
 Thomas> verifies that a Python interpreter is already available on the
 Thomas> build machine.

 Thomas> I believe this is something that should be clarified.

Yes, I agree it's a bit of a mess.

As we already build host-python as a dependency of python, we presumably
might as well use it for anything building python modules as well?


 >> ./output/host/usr/bin/python output/staging/usr/bin/python2.7-config --cflags
 >> -I/home/peko/source/buildroot/output/host/usr/include/python2.7 -I/home/peko/source/buildroot/output/host/usr/include/python2.7 -fno-strict-aliasing -O2 -I/home/peko/source/buildroot/output/host/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
 >> 
 >> E.G. your s/-I/-I$(STAGING_DIR)/ trick won't work (and it didn't work
 >> before if the host python wasn't built with --prefix=/usr).

 Thomas> Right, from the above, I am not sure we want to change the shebang of
 Thomas> this script.

Back when we dicussed host-python, somebody (I think it was Samuel)
mentioned that 'python' has become python 3.x on some distributions, and
as this script uses internal python stuff to spit out the needed CFLAGS
/ LDFLAGS, I think we NEED to run it with the correct python version.

Same for people with 'python' not in /usr.

The script apparently doesn't even work with python 3.x:

python3 /usr/bin/python2.7-config
  File "/usr/bin/python2.7-config", line 34
    print sysconfig.PREFIX
                  ^
SyntaxError: invalid syntax

-- 
Bye, Peter Korsgaard