Re: [CONSOLIDATED PULL 17/17] python: skip setup.py 'import check' when cross-compiling

[Tom Zanussi <tom.zanussi@intel.com>]

On Wed, 2011-11-09 at 02:34 -0800, Cui, Dexuan wrote:
> How I wish I could notice the patch this morning so I could save 1 day!
> 

??? Can you please explain what you mean?

> Anyway, thanks TomZ for the patch!  :-)
> 

You're welcome, I guess!

Tom

> Thanks,
> -- Dexuan
> 
> 
> > -----Original Message-----
> > From: openembedded-core-bounces@lists.openembedded.org
> > [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf
> > Of Saul Wold
> > Sent: Wednesday, November 09, 2011 3:23 AM
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [CONSOLIDATED PULL 17/17] python: skip setup.py
> > 'import check' when cross-compiling
> > 
> > From: Tom Zanussi <tom.zanussi@intel.com>
> > 
> > build_extension() in setup.py, as part of the build process, does an
> > 'import check' on the built extension.  The import check in turn
> > dlopen()'s the shared library associated with the extension, which
> > isn't something that makes sense if that library was cross-compiled
> > for a different architecture.
> > 
> > This was noticed with an x86_64 target that was compiled with avx
> > support, because it caused 'illegal instruction' exceptions:
> > 
> > | /bin/sh: line 1: 14575 Illegal instruction ... -E ./setup.py -q build
> > 
> > For other target architectures, it doesn't necessarily cause illegal
> > instruction exceptions, but still fails.  For example, on arm, the
> > failure pathway causes this warning:
> > 
> > *** WARNING: renaming "cmath" since importing it failed: .../cmath.so:
> >     wrong ELF class: ELFCLASS32
> > 
> > This patch to setup.py and the associated recipe changes allow the
> > whole 'import check' logic to be skipped when cross-compiling.
> > 
> > Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
> > ---
> >  .../python/setup_py_skip_cross_import_check.patch  |   27
> > ++++++++++++++++++++
> >  meta/recipes-devtools/python/python_2.7.2.bb       |    5 +++
> >  2 files changed, 32 insertions(+), 0 deletions(-)
> >  create mode 100644
> > meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.
> > patch
> > 
> > diff --git
> > a/meta/recipes-devtools/python/python/setup_py_skip_cross_import_chec
> > k.patch
> > b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_chec
> > k.patch
> > new file mode 100644
> > index 0000000..6ccdb94
> > --- /dev/null
> > +++
> > b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_chec
> > k.patch
> > @@ -0,0 +1,27 @@
> > +This patch skips over the 'import check' setup.py does when building
> > +extensions.  This generally won't work when cross-compiling.
> > +
> > +Upstream-Status: Inappropriate [embedded-specific]
> > +
> > +Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
> > +
> > +Index: Python-2.7.2/setup.py
> > +==========================================================
> > =========
> > +--- Python-2.7.2.orig/setup.py	2011-11-04 16:46:34.553796410 -0500
> > ++++ Python-2.7.2/setup.py	2011-11-04 16:59:49.692802313 -0500
> > +@@ -287,6 +287,15 @@
> > +                           (ext.name, sys.exc_info()[1]))
> > +             self.failed.append(ext.name)
> > +             return
> > ++
> > ++        # If we're cross-compiling, we want to skip the import check
> > ++        # i.e. we shouldn't be dynamically loading target shared libs
> > ++        if os.environ.get('CROSS_COMPILE') is not None:
> > ++            self.announce(
> > ++                'WARNING: skipping import check for cross-compiled
> > "%s"' %
> > ++                ext.name)
> > ++            return
> > ++
> > +         # Workaround for Mac OS X: The Carbon-based modules cannot
> > be
> > +         # reliably imported into a command-line Python
> > +         if 'Carbon' in ext.extra_link_args:
> > diff --git a/meta/recipes-devtools/python/python_2.7.2.bb
> > b/meta/recipes-devtools/python/python_2.7.2.bb
> > index bb088a4..360e64e 100644
> > --- a/meta/recipes-devtools/python/python_2.7.2.bb
> > +++ b/meta/recipes-devtools/python/python_2.7.2.bb
> > @@ -18,6 +18,7 @@ SRC_URI += "\
> >    file://multilib.patch \
> >    file://cgi_py.patch \
> >    file://remove_sqlite_rpath.patch \
> > +  file://setup_py_skip_cross_import_check.patch \
> >  "
> > 
> >  S = "${WORKDIR}/Python-${PV}"
> > @@ -57,6 +58,8 @@ do_compile() {
> >  	# then call do_install twice we get Makefile.orig == Makefile.sysroot
> >  	install -m 0644 Makefile Makefile.sysroot
> > 
> > +	export CROSS_COMPILE="${TARGET_PREFIX}"
> > +
> >  	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
> >  		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
> >  		STAGING_LIBDIR=${STAGING_LIBDIR} \
> > @@ -78,6 +81,8 @@ do_install() {
> >  	# make install needs the original Makefile, or otherwise the inclues
> > would
> >  	# go to ${D}${STAGING...}/...
> >  	install -m 0644 Makefile.orig Makefile
> > +
> > +	export CROSS_COMPILE="${TARGET_PREFIX}"
> > 
> >  	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
> >  		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
> > --
> > 1.7.6.4
> > 
> > 
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core