All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] python-{native}-2.6.5: Fix parallel build.
@ 2010-10-02 16:48 Paul Menzel
  2010-10-02 18:10 ` Khem Raj
  2010-10-03  2:03 ` Tom Rini
  0 siblings, 2 replies; 6+ messages in thread
From: Paul Menzel @ 2010-10-02 16:48 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 5393 bytes --]

Date: Sat, 2 Oct 2010 12:51:03 +0200

Randomly the build failed for me with the following error message.

    libpython2.6.so: undefined reference to `_PyParser_Grammar'

The applied patch in [5] tried to fix this problem, but it turns out that this was due to a possible race and already known in OpenOffice too [3].

R. David Murray (dmalcolm) <dmalcolm@fedoraproject.org> fixed this in Fedora [1] and I applied his patch.

I reported this issue upstream [4] and this will only be fixed in Python 2.7, so we have to keep this patch for Python 2.6.

[1] http://pkgs.fedoraproject.org/gitweb/?p=python.git;a=commitdiff;h=b95f6cc2ca6a009f97436c6aa16cfd70547353d9
[2] http://www.openoffice.org/issues/show_bug.cgi?id=114866
[3] http://doc.services.openoffice.org/wiki/RedTinderboxStatusInEIS
[4] http://bugs.python.org/issue10013
[5] http://patchwork.openembedded.org/patch/1899/

Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net>
---
 .../python/python-2.6.5/20-fix-parallel-make.patch |   40 ++++++++++++++++++++
 recipes/python/python-native_2.6.5.bb              |    2 +-
 recipes/python/python_2.6.5.bb                     |    5 +-
 3 files changed, 44 insertions(+), 3 deletions(-)
 create mode 100644 recipes/python/python-2.6.5/20-fix-parallel-make.patch

diff --git a/recipes/python/python-2.6.5/20-fix-parallel-make.patch b/recipes/python/python-2.6.5/20-fix-parallel-make.patch
new file mode 100644
index 0000000..1f7b24e
--- /dev/null
+++ b/recipes/python/python-2.6.5/20-fix-parallel-make.patch
@@ -0,0 +1,40 @@
+upstream: <http://bugs.python.org/issue10013>
+comment: As in the comment in the issue tracker this is fixed for py3k a little differently <http://svn.python.org/view?view=rev&revision=84068>. This patch is taken from Fedora.
+
+diff -up Python-2.7/Makefile.pre.in.fix-parallel-make Python-2.7/Makefile.pre.in
+--- Python-2.7/Makefile.pre.in.fix-parallel-make	2010-07-22 15:01:39.567996932 -0400
++++ Python-2.7/Makefile.pre.in	2010-07-22 15:47:02.437998509 -0400
+@@ -198,6 +198,7 @@ SIGNAL_OBJS=	@SIGNAL_OBJS@
+ 
+ ##########################################################################
+ # Grammar
++GRAMMAR_STAMP=	$(srcdir)/grammar-stamp
+ GRAMMAR_H=	$(srcdir)/Include/graminit.h
+ GRAMMAR_C=	$(srcdir)/Python/graminit.c
+ GRAMMAR_INPUT=	$(srcdir)/Grammar/Grammar
+@@ -515,10 +516,24 @@ Modules/getpath.o: $(srcdir)/Modules/get
+ Modules/python.o: $(srcdir)/Modules/python.c
+ 	$(MAINCC) -c $(PY_CFLAGS) -o $@ $(srcdir)/Modules/python.c
+ 
++# GNU "make" interprets rules with two dependents as two copies of the rule.
++# 
++# In a parallel build this can lead to pgen being run twice, once for each of
++# GRAMMAR_H and GRAMMAR_C, leading to race conditions in which the compiler
++# reads a partially-overwritten copy of one of these files, leading to syntax
++# errors (or linker errors if the fragment happens to be syntactically valid C)
++#
++# See http://www.gnu.org/software/hello/manual/automake/Multiple-Outputs.html
++# for more information.
++#
++# Introduce ".grammar-stamp" as a contrived single output from PGEN to avoid
++# this:
++$(GRAMMAR_H) $(GRAMMAR_C): $(GRAMMAR_STAMP)
+ 
+-$(GRAMMAR_H) $(GRAMMAR_C): $(PGEN) $(GRAMMAR_INPUT)
++$(GRAMMAR_STAMP): $(PGEN) $(GRAMMAR_INPUT)
+ 		-@$(INSTALL) -d Include
+ 		-$(HOSTPGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
++		touch $(GRAMMAR_STAMP)
+ 
+ $(PGEN):	$(PGENOBJS)
+ 		$(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
diff --git a/recipes/python/python-native_2.6.5.bb b/recipes/python/python-native_2.6.5.bb
index 76fbbb6..12c2252 100644
--- a/recipes/python/python-native_2.6.5.bb
+++ b/recipes/python/python-native_2.6.5.bb
@@ -11,11 +11,11 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.bz2 \
            file://10-distutils-fix-swig-parameter.patch \
            file://11-distutils-never-modify-shebang-line.patch \
            file://12-distutils-prefix-is-inside-staging-area.patch \
+           file://20-fix-parallel-make.patch
            file://debug.patch \
            file://nohostlibs.patch"
 SRC_URI[md5sum] = "6bef0417e71a1a1737ccf5750420fdb3"
 SRC_URI[sha256sum] = "62da62eb685621ede2be1275f11b89fa0e0be578db8daa5320d0a7855c0a9ebc"
-
 S = "${WORKDIR}/Python-${PV}"
 
 inherit native
diff --git a/recipes/python/python_2.6.5.bb b/recipes/python/python_2.6.5.bb
index ecd9f78..2c26512 100644
--- a/recipes/python/python_2.6.5.bb
+++ b/recipes/python/python_2.6.5.bb
@@ -3,7 +3,7 @@ DEPENDS = "python-native db gdbm openssl readline sqlite3 tcl zlib\
            ${@base_contains('DISTRO_FEATURES', 'tk', 'tk', '', d)}"
 DEPENDS_sharprom = "python-native db readline zlib gdbm openssl"
 # set to .0 on every increase of INC_PR
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
 
 SRC_URI = "\
   http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.bz2 \
@@ -14,8 +14,9 @@ SRC_URI = "\
   file://04-default-is-optimized.patch \
   file://05-enable-ctypes-cross-build.patch \
   file://06-ctypes-libffi-fix-configure.patch \
+  file://20-fix-parallel-make.patch \
   file://99-ignore-optimization-flag.patch \
-  file://sitecustomize.py \
+  file://sitecustomize.py
 "
 SRC_URI[md5sum] = "6bef0417e71a1a1737ccf5750420fdb3"
 SRC_URI[sha256sum] = "62da62eb685621ede2be1275f11b89fa0e0be578db8daa5320d0a7855c0a9ebc"
-- 
1.7.1


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH] python-{native}-2.6.5: Fix parallel build.
@ 2010-10-03 15:54 Paul Menzel
  2010-10-03 18:10 ` Paul Menzel
  0 siblings, 1 reply; 6+ messages in thread
From: Paul Menzel @ 2010-10-03 15:54 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 7771 bytes --]

Date: Sat, 2 Oct 2010 12:51:03 +0200

Randomly the build failed for me with the following error message.

    libpython2.6.so: undefined reference to `_PyParser_Grammar'

The applied patch in [5] tried to fix this problem, but it turns out that this also was due to a possible race. This is already known in OpenOffice too [3].

R. David Murray (dmalcolm) <dmalcolm@fedoraproject.org> fixed this in Fedora [1] and I applied his patch.

I reported this issue upstream [4] and this will only be fixed in Python 2.7, so we have to keep this patch for Python 2.6.

I put 00 in front of the name of the patch file because I did not want to rename the other patches.

I build tested this with `MACHINE = " beagleboard"` for `angstrom-2008.1` and `minimal-{eglibc,uclibc,libc}`.

[1] http://pkgs.fedoraproject.org/gitweb/?p=python.git;a=commitdiff;h=b95f6cc2ca6a009f97436c6aa16cfd70547353d9
[2] http://www.openoffice.org/issues/show_bug.cgi?id=114866
[3] http://doc.services.openoffice.org/wiki/RedTinderboxStatusInEIS
[4] http://bugs.python.org/issue10013
[5] http://patchwork.openembedded.org/patch/1899/

Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net>
---
v1 → v2: comments by Khem and Tom, add accidentely removed »\« at the line ending.
v2 → v3: Reorder patches, so that they can be applied to python and python-native.
---
 .../python/python-2.6.5/00-fix-parallel-make.patch |   40 ++++++++++++++++++++
 .../01-use-proper-tools-for-cross-build.patch      |   14 +++---
 recipes/python/python-native_2.6.5.bb              |    3 +-
 recipes/python/python_2.6.5.bb                     |    3 +-
 4 files changed, 51 insertions(+), 9 deletions(-)
 create mode 100644 recipes/python/python-2.6.5/00-fix-parallel-make.patch

diff --git a/recipes/python/python-2.6.5/00-fix-parallel-make.patch b/recipes/python/python-2.6.5/00-fix-parallel-make.patch
new file mode 100644
index 0000000..c80ca19
--- /dev/null
+++ b/recipes/python/python-2.6.5/00-fix-parallel-make.patch
@@ -0,0 +1,40 @@
+upstream: <http://bugs.python.org/issue10013>
+comment: As in the comment in the issue tracker this is fixed for py3k a little differently <http://svn.python.org/view?view=rev&revision=84068>. This patch is taken from Fedora.
+
+diff -up Python-2.7/Makefile.pre.in.fix-parallel-make Python-2.7/Makefile.pre.in
+--- Python-2.7/Makefile.pre.in.fix-parallel-make	2010-07-22 15:01:39.567996932 -0400
++++ Python-2.7/Makefile.pre.in	2010-07-22 15:47:02.437998509 -0400
+@@ -197,6 +197,7 @@ SIGNAL_OBJS=	@SIGNAL_OBJS@
+ 
+ ##########################################################################
+ # Grammar
++GRAMMAR_STAMP=	$(srcdir)/grammar-stamp
+ GRAMMAR_H=	$(srcdir)/Include/graminit.h
+ GRAMMAR_C=	$(srcdir)/Python/graminit.c
+ GRAMMAR_INPUT=	$(srcdir)/Grammar/Grammar
+@@ -514,10 +515,24 @@ Modules/getpath.o: $(srcdir)/Modules/get
+ Modules/python.o: $(srcdir)/Modules/python.c
+ 	$(MAINCC) -c $(PY_CFLAGS) -o $@ $(srcdir)/Modules/python.c
+ 
++# GNU "make" interprets rules with two dependents as two copies of the rule.
++# 
++# In a parallel build this can lead to pgen being run twice, once for each of
++# GRAMMAR_H and GRAMMAR_C, leading to race conditions in which the compiler
++# reads a partially-overwritten copy of one of these files, leading to syntax
++# errors (or linker errors if the fragment happens to be syntactically valid C)
++#
++# See http://www.gnu.org/software/hello/manual/automake/Multiple-Outputs.html
++# for more information.
++#
++# Introduce ".grammar-stamp" as a contrived single output from PGEN to avoid
++# this:
++$(GRAMMAR_H) $(GRAMMAR_C): $(GRAMMAR_STAMP)
+ 
+-$(GRAMMAR_H) $(GRAMMAR_C): $(PGEN) $(GRAMMAR_INPUT)
++$(GRAMMAR_STAMP): $(PGEN) $(GRAMMAR_INPUT)
+ 		-@$(INSTALL) -d Include
+ 		-$(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
++		touch $(GRAMMAR_STAMP)
+ 
+ $(PGEN):	$(PGENOBJS)
+ 		$(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
diff --git a/recipes/python/python-2.6.5/01-use-proper-tools-for-cross-build.patch b/recipes/python/python-2.6.5/01-use-proper-tools-for-cross-build.patch
index e89faa4..4b485ec 100644
--- a/recipes/python/python-2.6.5/01-use-proper-tools-for-cross-build.patch
+++ b/recipes/python/python-2.6.5/01-use-proper-tools-for-cross-build.patch
@@ -14,7 +14,7 @@ Index: Python-2.6.1/Makefile.pre.in
  
  # The task to run while instrument when building the profile-opt target
  PROFILE_TASK=	$(srcdir)/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck
-@@ -205,7 +206,7 @@ GRAMMAR_INPUT=	$(srcdir)/Grammar/Grammar
+@@ -206,7 +207,7 @@ GRAMMAR_INPUT=	$(srcdir)/Grammar/Grammar
  ##########################################################################
  # Parser
  PGEN=		Parser/pgen$(EXE)
@@ -23,7 +23,7 @@ Index: Python-2.6.1/Makefile.pre.in
  POBJS=		\
  		Parser/acceler.o \
  		Parser/grammar1.o \
-@@ -394,8 +395,8 @@ platform: $(BUILDPYTHON)
+@@ -395,8 +396,8 @@ platform: $(BUILDPYTHON)
  # Build the shared modules
  sharedmods: $(BUILDPYTHON)
  	@case $$MAKEFLAGS in \
@@ -34,16 +34,16 @@ Index: Python-2.6.1/Makefile.pre.in
  	esac
  
  # Build static library
-@@ -513,7 +514,7 @@ Modules/python.o: $(srcdir)/Modules/pyth
+@@ -531,7 +532,7 @@ $(GRAMMAR_H) $(GRAMMAR_C): $(GRAMMAR_STAMP)
  
- $(GRAMMAR_H) $(GRAMMAR_C): $(PGEN) $(GRAMMAR_INPUT)
+ $(GRAMMAR_STAMP): $(PGEN) $(GRAMMAR_INPUT)
  		-@$(INSTALL) -d Include
 -		-$(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
 +		-$(HOSTPGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
+ 		touch $(GRAMMAR_STAMP)
  
  $(PGEN):	$(PGENOBJS)
- 		$(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
-@@ -879,23 +880,23 @@ libinstall:	build_all $(srcdir)/Lib/$(PL
+@@ -902,23 +903,23 @@ libinstall:	build_all $(srcdir)/Lib/$(PL
  	done
  	$(INSTALL_DATA) $(srcdir)/LICENSE $(DESTDIR)$(LIBDEST)/LICENSE.txt
  	PYTHONPATH=$(DESTDIR)$(LIBDEST)  $(RUNSHARED) \
@@ -72,7 +72,7 @@ Index: Python-2.6.1/Makefile.pre.in
  
  # Create the PLATDIR source directory, if one wasn't distributed..
  $(srcdir)/Lib/$(PLATDIR):
-@@ -993,7 +994,7 @@ libainstall:	all
+@@ -1016,7 +1017,7 @@ libainstall:	all
  # Install the dynamically loadable modules
  # This goes into $(exec_prefix)
  sharedinstall:
diff --git a/recipes/python/python-native_2.6.5.bb b/recipes/python/python-native_2.6.5.bb
index 76fbbb6..51a1fb1 100644
--- a/recipes/python/python-native_2.6.5.bb
+++ b/recipes/python/python-native_2.6.5.bb
@@ -1,9 +1,10 @@
 require python.inc
 DEPENDS = "openssl-native bzip2-full-native zlib-native readline-native sqlite3-native"
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
 
 FILESPATHPKG .= ":python-${PV}:python"
 SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.bz2 \
+           file://00-fix-parallel-make.patch \
            file://00-fix-bindir-libdir-for-cross.patch \
            file://04-default-is-optimized.patch \
            file://05-enable-ctypes-cross-build.patch \
diff --git a/recipes/python/python_2.6.5.bb b/recipes/python/python_2.6.5.bb
index ecd9f78..c8393c1 100644
--- a/recipes/python/python_2.6.5.bb
+++ b/recipes/python/python_2.6.5.bb
@@ -3,10 +3,11 @@ DEPENDS = "python-native db gdbm openssl readline sqlite3 tcl zlib\
            ${@base_contains('DISTRO_FEATURES', 'tk', 'tk', '', d)}"
 DEPENDS_sharprom = "python-native db readline zlib gdbm openssl"
 # set to .0 on every increase of INC_PR
-PR = "${INC_PR}.0"
+PR = "${INC_PR}.1"
 
 SRC_URI = "\
   http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.bz2 \
+  file://00-fix-parallel-make.patch \
   file://00-fix-bindir-libdir-for-cross.patch \
   file://01-use-proper-tools-for-cross-build.patch \
   file://02-remove-test-for-cross.patch \
-- 
1.7.1

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply related	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2010-10-04  6:47 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-10-02 16:48 [PATCH] python-{native}-2.6.5: Fix parallel build Paul Menzel
2010-10-02 18:10 ` Khem Raj
2010-10-03  2:03 ` Tom Rini
  -- strict thread matches above, loose matches on Subject: below --
2010-10-03 15:54 Paul Menzel
2010-10-03 18:10 ` Paul Menzel
2010-10-04  6:47   ` Khem Raj

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.