[PATCH 0/6][v2] Add some intel-specific tune files, and related fixes

[PATCH 0/6][v2] Add some intel-specific tune files, and related fixes

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

This patchset adds some new tune files for corei7, ivb, and atom.

The corei7 and atom were build- and boot-tested on sugarbay (with
corei7-avx), jasperforest (corei7), crownbay, fri2, fishriver (atom).
The ivb is one I'll be using for new Ivy Bridge development and
hasn't been tested yet, but will be needed for that.

This patchset also contains a few patches for packages that had build
problems with the corei7-avx tuning.  The python and gmp packages had
problems because they tried to execute target code on the host and caused
illegal instruction errors.  The libzypp failed compilation because of
the hyphens in the tune name (corei7-avx).

v2: do it using create-pull-request, fix subject on python patch

The following changes since commit 28940eb97d835b45ecbf832f25e94fc1574f25a1:
  Koen Kooi (1):
        lighttpd 1.4.29: rename index.html to avoid clashes with DL_DIR

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib.git tzanussi/newtune-v0
  http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/newtune-v0

Tom Zanussi (6):
  tune-corei7: Add tune file for Intel Core i7 machines
  tune-ivb: Add tune file for Intel Ivy Bridge machines
  machine/include/tune-atom.inc: add atom-specific settings
  gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC
  libzypp: fix mishandling of hyphenated arches
  python: skip setup.py 'import check' when cross-compiling

 meta/conf/machine/include/tune-atom.inc            |   26 +++++++++++++++++-
 meta/conf/machine/include/tune-corei7.inc          |   17 ++++++++++++
 meta/conf/machine/include/tune-ivb.inc             |   11 ++++++++
 .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
 meta/recipes-devtools/python/python_2.7.2.bb       |    5 +++
 meta/recipes-extended/libzypp/libzypp_git.bb       |    8 +++---
 meta/recipes-support/gmp/gmp_5.0.2.bb              |    3 +-
 7 files changed, 90 insertions(+), 7 deletions(-)
 create mode 100644 meta/conf/machine/include/tune-corei7.inc
 create mode 100644 meta/conf/machine/include/tune-ivb.inc
 create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch

[PATCH 1/6] tune-corei7: Add tune file for Intel Core i7 machines

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

Add a tune file for Intel Core i7 machines both with and without AVX
support.

Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
---
 meta/conf/machine/include/tune-corei7.inc |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)
 create mode 100644 meta/conf/machine/include/tune-corei7.inc

diff --git a/meta/conf/machine/include/tune-corei7.inc b/meta/conf/machine/include/tune-corei7.inc
new file mode 100644
index 0000000..2e0c67f
--- /dev/null
+++ b/meta/conf/machine/include/tune-corei7.inc
@@ -0,0 +1,17 @@
+DEFAULTTUNE ?= "corei7"
+TUNE_PKGARCH ?= "${@bb.utils.contains("TUNE_FEATURES", "avx", "corei7-avx", "corei7", d)}"
+
+require conf/machine/include/tune-x86_64.inc
+
+TUNEVALID[corei7] = "Enable corei7-specific processor optimizations"
+TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "corei7", "-march=corei7${COREI7_AVX}", "", d)}"
+
+COREI7_AVX = "${@bb.utils.contains("TUNE_FEATURES", [ "corei7", "avx" ], "-avx", "", d)}"
+
+AVAILTUNES += "corei7"
+TUNE_FEATURES_tune-corei7 = "${TUNE_FEATURES_tune-x86-64} corei7"
+PACKAGE_EXTRA_ARCHS_tune-corei7 = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} corei7"
+
+AVAILTUNES += "corei7-avx"
+TUNE_FEATURES_tune-corei7-avx = "${TUNE_FEATURES_tune-x86-64} corei7 avx"
+PACKAGE_EXTRA_ARCHS_tune-corei7-avx = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} corei7-avx"
-- 
1.7.0.4

[PATCH 2/6] tune-ivb: Add tune file for Intel Ivy Bridge machines

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

Add a tune file for Intel Ivy Bridge machines.

Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
---
 meta/conf/machine/include/tune-ivb.inc |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)
 create mode 100644 meta/conf/machine/include/tune-ivb.inc

diff --git a/meta/conf/machine/include/tune-ivb.inc b/meta/conf/machine/include/tune-ivb.inc
new file mode 100644
index 0000000..3c0b670
--- /dev/null
+++ b/meta/conf/machine/include/tune-ivb.inc
@@ -0,0 +1,11 @@
+DEFAULTTUNE ?= "ivb"
+TUNE_PKGARCH ?= "${@bb.utils.contains("TUNE_FEATURES", "ivb", "ivb", "", d)}"
+
+require conf/machine/include/tune-x86_64.inc
+
+TUNEVALID[ivb] = "Enable ivb-specific processor optimizations"
+TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "ivb", "-march=core-avx-i", "", d)}"
+
+AVAILTUNES += "ivb"
+TUNE_FEATURES_tune-ivb = "${TUNE_FEATURES_tune-x86-64} ivb"
+PACKAGE_EXTRA_ARCHS_tune-ivb = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} ivb"
-- 
1.7.0.4

[PATCH 3/6] machine/include/tune-atom.inc: add atom-specific settings

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

Atom tunings are currently the same as core2; this patch changes them
to make use of atom-specific settings such as the atom arch and
instruction support in gcc.

Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
---
 meta/conf/machine/include/tune-atom.inc |   26 ++++++++++++++++++++++++--
 1 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/meta/conf/machine/include/tune-atom.inc b/meta/conf/machine/include/tune-atom.inc
index 5e1bb74..ab4aa14 100644
--- a/meta/conf/machine/include/tune-atom.inc
+++ b/meta/conf/machine/include/tune-atom.inc
@@ -1,2 +1,24 @@
-# Atom tunings are the same as core2 for now...
-require conf/machine/include/tune-core2.inc
+DEFAULTTUNE ?= "atom"
+TUNE_PKGARCH ?= "${@bb.utils.contains("TUNE_FEATURES", "m32", "atom", "atom-64", d)}"
+
+require conf/machine/include/tune-i586.inc
+
+# Extra tune features
+TUNEVALID[atom] = "Enable atom-specific processor optimizations"
+TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "atom", "-march=atom -mssse3 -mfpmath=sse", "", d)}"
+
+# Extra tune selections
+AVAILTUNES += "atom"
+TUNE_FEATURES_tune-atom ?= "${TUNE_FEATURES_tune-x86} atom"
+BASE_LIB_tune-atom ?= "lib"
+PACKAGE_EXTRA_ARCHS_tune-atom = "${PACKAGE_EXTRA_ARCHS_tune-x86} i386 i486 i586 i686 atom"
+
+AVAILTUNES += "atom-64"
+TUNE_FEATURES_tune-atom-64 ?= "${TUNE_FEATURES_tune-x86-64} atom"
+BASE_LIB_tune-atom-64 ?= "lib64"
+PACKAGE_EXTRA_ARCHS_tune-atom-64 = "${PACKAGE_EXTRA_ARCHS_tune-x86-64} atom-64"
+
+AVAILTUNES += "atom-64-x32"
+TUNE_FEATURES_tune-atom-64-x32 ?= "${TUNE_FEATURES_tune-x86-64-x32} atom"
+BASE_LIB_tune-atom-64-x32 ?= "libx32"
+PACKAGE_EXTRA_ARCHS_tune-atom-64-x32 = "${PACKAGE_EXTRA_ARCHS_tune-x86-64-x32} atom-64-x32"
-- 
1.7.0.4

[PATCH 4/6] gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

CC_FOR_BUILD was compiling the test programs using the target's
compile options and executing those on the host, causing errors such
as:

/bin/sh: line 1: 15032 Illegal instruction     ./gen-bases table 64 0 > mpn/mp_bases.c
/bin/sh: line 1: 15033 Illegal instruction     ./gen-bases header 64 0 > mp_bases.h

Export CC_FOR_BUILD using BUILD_CC to fix the problem.

Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
---
 meta/recipes-support/gmp/gmp_5.0.2.bb |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/meta/recipes-support/gmp/gmp_5.0.2.bb b/meta/recipes-support/gmp/gmp_5.0.2.bb
index 03fef45..16bdcbc 100644
--- a/meta/recipes-support/gmp/gmp_5.0.2.bb
+++ b/meta/recipes-support/gmp/gmp_5.0.2.bb
@@ -2,11 +2,12 @@ require gmp.inc
 LICENSE="LGPLv3&GPLv3"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
 		    file://version.c;endline=18;md5=d8c56b52b9092346b9f93b4da65ef790"
-PR = "r0"
+PR = "r1"
 
 SRC_URI_append += "file://sh4-asmfix.patch \
                    file://use-includedir.patch "
 
+export CC_FOR_BUILD = "${BUILD_CC}"
 
 SRC_URI[md5sum] = "0bbaedc82fb30315b06b1588b9077cd3"
 SRC_URI[sha256sum] = "dbc2db76fdd4e99f85d5e35aa378ed62c283e0d586b91bd8703aff75a7804c28"
-- 
1.7.0.4

[PATCH 5/6] libzypp: fix mishandling of hyphenated arches

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

Several hyphen-to-underscore translations were missing, causing
compiler errors trying to build arches with hyphens in their names.
This adds the missing translations.

Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
---
 meta/recipes-extended/libzypp/libzypp_git.bb |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-extended/libzypp/libzypp_git.bb b/meta/recipes-extended/libzypp/libzypp_git.bb
index 20e8f7c..a6b270c 100644
--- a/meta/recipes-extended/libzypp/libzypp_git.bb
+++ b/meta/recipes-extended/libzypp/libzypp_git.bb
@@ -11,7 +11,7 @@ DEPENDS  = "rpm boost curl libxml2 zlib sat-solver expat openssl udev"
 S = "${WORKDIR}/git"
 SRCREV = "15b6c52260bbc52b3d8e585e271b67e10cc7c433"
 PV = "0.0-git${SRCPV}"
-PR = "r15"
+PR = "r16"
 
 SRC_URI = "git://github.com/openSUSE/libzypp.git;protocol=git \
            file://no-doc.patch \
@@ -98,10 +98,10 @@ do_archgen () {
 		esac
 		if [ "${AVOID_CONSTRUCTOR}" != "true" ]; then
 		  echo -n "  const Arch Arch_${each_arch} " | tr - _		>> zypp/oe-arch.h
-		  echo "(_${each_arch});" 					>> zypp/oe-arch.h
+		  echo "(_${each_arch});" | tr - _				>> zypp/oe-arch.h
 		else
 		  echo -n "  const Arch Arch_${each_arch} " | tr - _		>> zypp/oe-arch.h
-		  echo "( IdString ( \"${each_arch}\" ) );" 			>> zypp/oe-arch.h
+		  echo "( IdString ( \"${each_arch}\" ) );" | tr - _		>> zypp/oe-arch.h
 		fi
 	done
 	echo "#endif /* OE_PROTO */"						>> zypp/oe-arch.h
@@ -142,7 +142,7 @@ do_archgen () {
 		COMPAT_WITH="${CARCH},${COMPAT} $COMPAT_WITH"
 	done
 	for each_compat in ${COMPAT_WITH} ; do
-		echo "        defCompatibleWith( ${each_compat} );"		>> zypp/oe-arch.h
+		echo "        defCompatibleWith( ${each_compat} );" | tr - _	>> zypp/oe-arch.h
 	done
 	echo "#endif /* DEF_COMPAT */"						>> zypp/oe-arch.h
 	echo ""									>> zypp/oe-arch.h
-- 
1.7.0.4

[PATCH 6/6] python: skip setup.py 'import check' when cross-compiling

[tom.zanussi]

From: Tom Zanussi <tom.zanussi@intel.com>

build_extension() in setup.py, as part of the build process, does an
'import check' on the built extension.  The import check in turn
dlopen()'s the shared library associated with the extension, which
isn't something that makes sense if that library was cross-compiled
for a different architecture.

This was noticed with an x86_64 target that was compiled with avx
support, because it caused 'illegal instruction' exceptions:

| /bin/sh: line 1: 14575 Illegal instruction ... -E ./setup.py -q build

For other target architectures, it doesn't necessarily cause illegal
instruction exceptions, but still fails.  For example, on arm, the
failure pathway causes this warning:

*** WARNING: renaming "cmath" since importing it failed: .../cmath.so:
    wrong ELF class: ELFCLASS32

This patch to setup.py and the associated recipe changes allow the
whole 'import check' logic to be skipped when cross-compiling.

Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
---
 .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
 meta/recipes-devtools/python/python_2.7.2.bb       |    5 +++
 2 files changed, 32 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch

diff --git a/meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
new file mode 100644
index 0000000..6ccdb94
--- /dev/null
+++ b/meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
@@ -0,0 +1,27 @@
+This patch skips over the 'import check' setup.py does when building
+extensions.  This generally won't work when cross-compiling.
+
+Upstream-Status: Inappropriate [embedded-specific]
+
+Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
+
+Index: Python-2.7.2/setup.py
+===================================================================
+--- Python-2.7.2.orig/setup.py	2011-11-04 16:46:34.553796410 -0500
++++ Python-2.7.2/setup.py	2011-11-04 16:59:49.692802313 -0500
+@@ -287,6 +287,15 @@
+                           (ext.name, sys.exc_info()[1]))
+             self.failed.append(ext.name)
+             return
++
++        # If we're cross-compiling, we want to skip the import check
++        # i.e. we shouldn't be dynamically loading target shared libs
++        if os.environ.get('CROSS_COMPILE') is not None:
++            self.announce(
++                'WARNING: skipping import check for cross-compiled "%s"' %
++                ext.name)
++            return
++
+         # Workaround for Mac OS X: The Carbon-based modules cannot be
+         # reliably imported into a command-line Python
+         if 'Carbon' in ext.extra_link_args:
diff --git a/meta/recipes-devtools/python/python_2.7.2.bb b/meta/recipes-devtools/python/python_2.7.2.bb
index bb088a4..360e64e 100644
--- a/meta/recipes-devtools/python/python_2.7.2.bb
+++ b/meta/recipes-devtools/python/python_2.7.2.bb
@@ -18,6 +18,7 @@ SRC_URI += "\
   file://multilib.patch \
   file://cgi_py.patch \
   file://remove_sqlite_rpath.patch \
+  file://setup_py_skip_cross_import_check.patch \
 "
 
 S = "${WORKDIR}/Python-${PV}"
@@ -57,6 +58,8 @@ do_compile() {
 	# then call do_install twice we get Makefile.orig == Makefile.sysroot
 	install -m 0644 Makefile Makefile.sysroot
 
+	export CROSS_COMPILE="${TARGET_PREFIX}"
+
 	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
 		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
 		STAGING_LIBDIR=${STAGING_LIBDIR} \
@@ -78,6 +81,8 @@ do_install() {
 	# make install needs the original Makefile, or otherwise the inclues would
 	# go to ${D}${STAGING...}/...
 	install -m 0644 Makefile.orig Makefile
+
+	export CROSS_COMPILE="${TARGET_PREFIX}"
 	
 	oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/pgen \
 		HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python \
-- 
1.7.0.4

Re: [PATCH 0/6][v2] Add some intel-specific tune files, and related fixes

[Saul Wold]

On 11/07/2011 12:28 PM, tom.zanussi@intel.com wrote:
> From: Tom Zanussi<tom.zanussi@intel.com>
>
> This patchset adds some new tune files for corei7, ivb, and atom.
>
> The corei7 and atom were build- and boot-tested on sugarbay (with
> corei7-avx), jasperforest (corei7), crownbay, fri2, fishriver (atom).
> The ivb is one I'll be using for new Ivy Bridge development and
> hasn't been tested yet, but will be needed for that.
>
> This patchset also contains a few patches for packages that had build
> problems with the corei7-avx tuning.  The python and gmp packages had
> problems because they tried to execute target code on the host and caused
> illegal instruction errors.  The libzypp failed compilation because of
> the hyphens in the tune name (corei7-avx).
>
> v2: do it using create-pull-request, fix subject on python patch
>
> The following changes since commit 28940eb97d835b45ecbf832f25e94fc1574f25a1:
>    Koen Kooi (1):
>          lighttpd 1.4.29: rename index.html to avoid clashes with DL_DIR
>
> are available in the git repository at:
>
>    git://git.yoctoproject.org/poky-contrib.git tzanussi/newtune-v0
>    http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=tzanussi/newtune-v0
>
Tom,

Thank you, this makes it much easier to pull and merge into the testing 
branch that I work with.

Sau!

> Tom Zanussi (6):
>    tune-corei7: Add tune file for Intel Core i7 machines
>    tune-ivb: Add tune file for Intel Ivy Bridge machines
>    machine/include/tune-atom.inc: add atom-specific settings
>    gmp_5.0.2: Set CC_FOR_BUILD to BUILD_CC
>    libzypp: fix mishandling of hyphenated arches
>    python: skip setup.py 'import check' when cross-compiling
>
>   meta/conf/machine/include/tune-atom.inc            |   26 +++++++++++++++++-
>   meta/conf/machine/include/tune-corei7.inc          |   17 ++++++++++++
>   meta/conf/machine/include/tune-ivb.inc             |   11 ++++++++
>   .../python/setup_py_skip_cross_import_check.patch  |   27 ++++++++++++++++++++
>   meta/recipes-devtools/python/python_2.7.2.bb       |    5 +++
>   meta/recipes-extended/libzypp/libzypp_git.bb       |    8 +++---
>   meta/recipes-support/gmp/gmp_5.0.2.bb              |    3 +-
>   7 files changed, 90 insertions(+), 7 deletions(-)
>   create mode 100644 meta/conf/machine/include/tune-corei7.inc
>   create mode 100644 meta/conf/machine/include/tune-ivb.inc
>   create mode 100644 meta/recipes-devtools/python/python/setup_py_skip_cross_import_check.patch
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>