[PATCH 0/6 V3] Share gcc work directories

[PATCH 0/6 V3] Share gcc work directories

[Robert Yang]

Changes of V3:

1) Use --with-headers to replace the:

   sed -i 's:^\([ ]*\)glibc_header_dir=\"${with_build_sysroot}/usr/include\": ...

2) Fix the build of gcc-cross-canadian

3) Fix gcc-crosssdk which make the source incompatible during its
   building.

4) Test "bitbake meta-toolchain", and also do run test for the
   pokysdk(unpack the toolchain to /opt, setup the env by sourcing
   environment-setup-armv5te-poky-linux-gnueabi, and build some pkgs
   out of poky, then use command file to check the binary file type.

5) Make sure that no gcc build would make the source incompatible,
   except gcc-cross_csl-arm-2008q1.bb, but I think this is fine, its
   BINV = "4.2.3", not 4.5.1 or 4.6.0.

Thanks for Richard's great help.

// Robert


The following changes since commit 2163461ec94528ecf046a04edc5db3d2dd3a6b8b:

  systemtap: remove non-core COMPATIBLE_MACHINES (2011-06-16 22:14:06 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib robert/share_gcc
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/share_gcc

Robert Yang (6):
  bitbake: share source directory
  Share gcc work directories
  gcc-4.5.1: share work directories
  gcc-4.6.0: share work directories
  gcc-crosssdk: share work directories
  gcc-cross-canadian: share work directories

 bitbake/lib/bb/build.py                            |    4 +-
 bitbake/lib/bb/cache.py                            |    3 +
 bitbake/lib/bb/runqueue.py                         |   10 +++
 meta/recipes-devtools/gcc/gcc-4.5.1.inc            |    1 +
 .../gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch   |   57 ++++++++++++++
 meta/recipes-devtools/gcc/gcc-4.6.0.inc            |    1 +
 .../gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch   |   80 ++++++++++++++++++++
 meta/recipes-devtools/gcc/gcc-common.inc           |   32 +++++++-
 meta/recipes-devtools/gcc/gcc-configure-common.inc |   33 ++++-----
 meta/recipes-devtools/gcc/gcc-configure-cross.inc  |    4 +-
 meta/recipes-devtools/gcc/gcc-configure-sdk.inc    |    8 ++
 meta/recipes-devtools/gcc/gcc-crosssdk.inc         |   13 +++-
 12 files changed, 220 insertions(+), 26 deletions(-)
 create mode 100644 meta/recipes-devtools/gcc/gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch
 create mode 100644 meta/recipes-devtools/gcc/gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch

[PATCH 1/6 V3] bitbake: share source directory

[Robert Yang]

This patch is derived from Richard, it is a quick proof of concept to
show how source code could be shared between recipes which use ${B} to
have a separate build directory compared to source directory ${S}.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/lib/bb/build.py    |    4 ++--
 bitbake/lib/bb/cache.py    |    3 +++
 bitbake/lib/bb/runqueue.py |   10 ++++++++++
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
index 7947906..052a3bf 100644
--- a/bitbake/lib/bb/build.py
+++ b/bitbake/lib/bb/build.py
@@ -383,10 +383,10 @@ def stamp_internal(taskname, d, file_name):
         taskflagname = taskname.replace("_setscene", "")
 
     if file_name:
-        stamp = d.stamp[file_name]
+        stamp = d.stamp_base[file_name].get(taskflagname) or d.stamp[file_name]
         extrainfo = d.stamp_extrainfo[file_name].get(taskflagname) or ""
     else:
-        stamp = d.getVar('STAMP', True)
+        stamp = d.getVarFlag(taskflagname, 'stamp-base', True) or d.getVar('STAMP', True)
         file_name = d.getVar('BB_FILENAME', True)
         extrainfo = d.getVarFlag(taskflagname, 'stamp-extra-info', True) or ""
 
diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py
index 6c92a93..99d7395 100644
--- a/bitbake/lib/bb/cache.py
+++ b/bitbake/lib/bb/cache.py
@@ -124,6 +124,7 @@ class CoreRecipeInfo(RecipeInfoCommon):
         self.broken = self.getvar('BROKEN', metadata)
         self.not_world = self.getvar('EXCLUDE_FROM_WORLD', metadata)
         self.stamp = self.getvar('STAMP', metadata)
+        self.stamp_base = self.flaglist('stamp-base', self.tasks, metadata)
         self.stamp_extrainfo = self.flaglist('stamp-extra-info', self.tasks, metadata)
         self.packages_dynamic = self.listvar('PACKAGES_DYNAMIC', metadata)
         self.depends          = self.depvar('DEPENDS', metadata)
@@ -151,6 +152,7 @@ class CoreRecipeInfo(RecipeInfoCommon):
         cachedata.pkg_dp = {}
 
         cachedata.stamp = {}
+        cachedata.stamp_base = {}
         cachedata.stamp_extrainfo = {}
         cachedata.fn_provides = {}
         cachedata.pn_provides = defaultdict(list)
@@ -183,6 +185,7 @@ class CoreRecipeInfo(RecipeInfoCommon):
         cachedata.pkg_pepvpr[fn] = (self.pe, self.pv, self.pr)
         cachedata.pkg_dp[fn] = self.defaultpref
         cachedata.stamp[fn] = self.stamp
+        cachedata.stamp_base[fn] = self.stamp_base
         cachedata.stamp_extrainfo[fn] = self.stamp_extrainfo
 
         provides = [self.pn]
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 7a17fce..a4be099 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -105,6 +105,11 @@ class RunQueueScheduler(object):
             if self.rq.runq_running[taskid] == 1:
                 continue
             if self.rq.runq_buildable[taskid] == 1:
+                fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[taskid]]
+                taskname = self.rqdata.runq_task[taskid]
+                stamp = bb.build.stampfile(taskname, self.rqdata.dataCache, fn)
+                if stamp in self.rq.build_stamps.values():
+                    continue
                 return taskid
 
     def next(self):
@@ -1009,6 +1014,7 @@ class RunQueueExecute:
         self.runq_complete = []
         self.build_pids = {}
         self.build_pipes = {}
+        self.build_stamps = {}
         self.failed_fnids = []
 
     def runqueue_process_waitpid(self):
@@ -1023,6 +1029,9 @@ class RunQueueExecute:
         del self.build_pids[result[0]]
         self.build_pipes[result[0]].close()
         del self.build_pipes[result[0]]
+        # self.build_stamps[result[0]] may not exist when use shared work directory.
+        if result[0] in self.build_stamps.keys():
+            del self.build_stamps[result[0]]
         if result[1] != 0:
             self.task_fail(task, result[1]>>8)
         else:
@@ -1319,6 +1328,7 @@ class RunQueueExecuteTasks(RunQueueExecute):
 
             self.build_pids[pid] = task
             self.build_pipes[pid] = runQueuePipe(pipein, pipeout, self.cfgData)
+            self.build_stamps[pid] = bb.build.stampfile(taskname, self.rqdata.dataCache, fn)
             self.runq_running[task] = 1
             self.stats.taskActive()
             if self.stats.active < self.number_tasks:
-- 
1.7.1

[PATCH 2/6 V3] Share gcc work directories

[Robert Yang]

This patched is derived from Richard, make gcc use the shared source
directory during the different building:

1) Make gcc-cross, gcc-cross-initial, gcc-cross-intermediate and
   gcc-runtime share the same source directory.

2) The source directory is ${TMPDIR}/work-shared/gcc-${PV}, for example:
   tmp/work-shared/gcc-4.6.0

3) Fix do_clean to clean the shared source directory and stamps

4) gcc uses sed and creates config files against ${S} which means the
   directory should not be shared. Change the way to make it work:

   * The configure option --with-headers=${STAGING_DIR_TARGET}${SYSTEMHEADERS}
     can replace the sed command, see the code in configure:

        if test "x$with_headers" != x; then
          glibc_header_dir=$with_headers

    This has the same effect as the sed command:

    sed -i 's:^\([  ]*\)glibc_header_dir=\"${with_build_sysroot}/usr/include\": ...

    so add the --with-headers=${STAGING_DIR_TARGET}${SYSTEMHEADERS} to
    gcc-configure-cross.inc( not add to gcc-configure-common.inc, since
    not all the gcc building need this, the one which has its own do_configure
    doesn't need it).

   * Move t-oe from ${T} to ${B}/gcc, so that the patched Makefile.in
     can read it easily, please see the commit for gcc-4.5.1 and
     gcc-4.6.0.

   * Use the defaults.h in ${B}/gcc instead of ${S}/gcc, and the patched
     configure.ac(configure) can read it correctly, please see the
     commit for gcc-4.5.1 and gcc-4.6.0.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-common.inc           |   32 ++++++++++++++++++-
 meta/recipes-devtools/gcc/gcc-configure-common.inc |   33 ++++++++-----------
 meta/recipes-devtools/gcc/gcc-configure-cross.inc  |    4 ++-
 3 files changed, 47 insertions(+), 22 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-common.inc b/meta/recipes-devtools/gcc/gcc-common.inc
index a3fa234..7bf036c 100644
--- a/meta/recipes-devtools/gcc/gcc-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-common.inc
@@ -37,10 +37,38 @@ ${GNU_MIRROR}/gcc/	http://gcc.get-software.com/releases/ \n \
 #
 gcclibdir = "${libdir}/gcc"
 BINV = "${PV}"
-S = "${WORKDIR}/gcc-${PV}"
-B = "${S}/build.${HOST_SYS}.${TARGET_SYS}"
+#S = "${WORKDIR}/gcc-${PV}"
+S = "${TMPDIR}/work-shared/gcc-${PV}/gcc-${PV}"
+B = "${WORKDIR}/gcc-${PV}/build.${HOST_SYS}.${TARGET_SYS}"
+
+# SS means Shared Stamps directory
+SS = "${TMPDIR}/stamps/work-shared/gcc-${PV}"
+do_fetch[stamp-base] = "${SS}"
+do_unpack[stamp-base] = "${SS}"
+do_patch[stamp-base] = "${SS}"
+
+# SW means Shared Work directory
+SW = "${TMPDIR}/work-shared/gcc-${PV}"
+WORKDIR_task-unpack = "${SW}"
+WORKDIR_task-patch = "${SW}"
 
 target_includedir ?= "${includedir}"
 target_libdir ?= "${libdir}"
 target_base_libdir ?= "${base_libdir}"
 target_prefix ?= "${prefix}"
+
+CLEANFUNCS += "workshared_clean"
+# The do_clean should be exclusive since share ${S}
+do_clean[lockfiles] = "${TMPDIR}/stamps/work-shared/gcc-${PV}.clean.lock"
+
+python workshared_clean () {
+	"""clear the source directory"""
+	dir = bb.data.expand("${SW}", d)
+	bb.note("Removing " + dir)
+	oe.path.remove(dir)
+
+	"""clear the the stamps in work-shared"""
+	dir = "%s.*" % bb.data.expand(d.getVarFlag('do_fetch', 'stamp-base', True), d)
+	bb.note("Removing " + dir)
+	oe.path.remove(dir)
+}
diff --git a/meta/recipes-devtools/gcc/gcc-configure-common.inc b/meta/recipes-devtools/gcc/gcc-configure-common.inc
index f7b5836..c543aae 100644
--- a/meta/recipes-devtools/gcc/gcc-configure-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-configure-common.inc
@@ -88,25 +88,20 @@ do_configure () {
 	(cd ${S} && gnu-configize) || die "failure running gnu-configize"
 	
 	# teach gcc to find correct target includedir when checking libc ssp support
-	sed -i 's:^\([ 	]*\)glibc_header_dir=\"${with_build_sysroot}/usr/include\":\1glibc_header_dir=\"${with_build_sysroot}${SYSTEMHEADERS}\":g' ${S}/gcc/configure.ac
-	sed -i 's:^\([ 	]*\)glibc_header_dir=\"${with_build_sysroot}/usr/include\":\1glibc_header_dir=\"${with_build_sysroot}${SYSTEMHEADERS}\":g' ${S}/gcc/configure
-
-	# splice our idea of where the headers live into gcc's world
-	echo "NATIVE_SYSTEM_HEADER_DIR = ${SYSTEMHEADERS}" > ${T}/t-oe
-	sed 's%^tmake_file=.*$%& ${T}/t-oe%' < ${S}/gcc/Makefile.in >${S}/gcc/Makefile.in.new
-	mv ${S}/gcc/Makefile.in.new ${S}/gcc/Makefile.in
-	cat ${S}/gcc/defaults.h | grep -v "\#endif.*GCC_DEFAULTS_H" > ${S}/gcc/defaults.h.new
-	echo "#ifndef STANDARD_INCLUDE_DIR" >> ${S}/gcc/defaults.h.new
-	echo "#define STANDARD_INCLUDE_DIR \"${SYSTEMHEADERS}\"" >> ${S}/gcc/defaults.h.new
-	echo "#endif" >> ${S}/gcc/defaults.h.new
-	echo "#ifndef STANDARD_STARTFILE_PREFIX_1" >> ${S}/gcc/defaults.h.new
-	echo "#define STANDARD_STARTFILE_PREFIX_1 \"${SYSTEMLIBS}\"" >> ${S}/gcc/defaults.h.new
-	echo "#endif" >> ${S}/gcc/defaults.h.new
-	echo "#ifndef STANDARD_STARTFILE_PREFIX_2" >> ${S}/gcc/defaults.h.new
-	echo "#define STANDARD_STARTFILE_PREFIX_2 \"${SYSTEMLIBS1}\"" >> ${S}/gcc/defaults.h.new
-	echo "#endif" >> ${S}/gcc/defaults.h.new
-	echo "#endif /* ! GCC_DEFAULTS_H */" >> ${S}/gcc/defaults.h.new
-	mv ${S}/gcc/defaults.h.new ${S}/gcc/defaults.h
+	mkdir -p ${B}/gcc
+	echo "NATIVE_SYSTEM_HEADER_DIR = ${SYSTEMHEADERS}" > ${B}/gcc/t-oe
+	cat ${S}/gcc/defaults.h | grep -v "\#endif.*GCC_DEFAULTS_H" > ${B}/gcc/defaults.h.new
+	echo "#ifndef STANDARD_INCLUDE_DIR" >> ${B}/gcc/defaults.h.new
+	echo "#define STANDARD_INCLUDE_DIR \"${SYSTEMHEADERS}\"" >> ${B}/gcc/defaults.h.new
+	echo "#endif" >> ${B}/gcc/defaults.h.new
+	echo "#ifndef STANDARD_STARTFILE_PREFIX_1" >> ${B}/gcc/defaults.h.new
+	echo "#define STANDARD_STARTFILE_PREFIX_1 \"${SYSTEMLIBS}\"" >> ${B}/gcc/defaults.h.new
+	echo "#endif" >> ${B}/gcc/defaults.h.new
+	echo "#ifndef STANDARD_STARTFILE_PREFIX_2" >> ${B}/gcc/defaults.h.new
+	echo "#define STANDARD_STARTFILE_PREFIX_2 \"${SYSTEMLIBS1}\"" >> ${B}/gcc/defaults.h.new
+	echo "#endif" >> ${B}/gcc/defaults.h.new
+	echo "#endif /* ! GCC_DEFAULTS_H */" >> ${B}/gcc/defaults.h.new
+	mv ${B}/gcc/defaults.h.new ${B}/gcc/defaults.h
 	oe_runconf
 }
 
diff --git a/meta/recipes-devtools/gcc/gcc-configure-cross.inc b/meta/recipes-devtools/gcc/gcc-configure-cross.inc
index 346d164..36edb81 100644
--- a/meta/recipes-devtools/gcc/gcc-configure-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-configure-cross.inc
@@ -2,7 +2,9 @@ require gcc-configure-common.inc
 
 USE_NLS = '${@base_conditional( "TARGET_OS", "linux-uclibc", "no", "", d )}'
 
-EXTRA_OECONF += " --enable-poison-system-directories "
+EXTRA_OECONF += " --enable-poison-system-directories \
+		  --with-headers=${STAGING_DIR_TARGET}${SYSTEMHEADERS} \
+		"
 
 EXTRA_OECONF_PATHS = "--with-local-prefix=${STAGING_DIR_TARGET}${target_exec_prefix} \
 		      --with-gxx-include-dir=${target_includedir}/c++ \
-- 
1.7.1

[PATCH 3/6 V3] gcc-4.5.1: share work directories

[Robert Yang]

Fix configure and Makefile to read the defaults.h and t-oe from ${B},
so that the ${S} can be shared.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-4.5.1.inc            |    1 +
 .../gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch   |   57 ++++++++++++++++++++
 2 files changed, 58 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-devtools/gcc/gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch

diff --git a/meta/recipes-devtools/gcc/gcc-4.5.1.inc b/meta/recipes-devtools/gcc/gcc-4.5.1.inc
index 3c4902e..2569e2e 100644
--- a/meta/recipes-devtools/gcc/gcc-4.5.1.inc
+++ b/meta/recipes-devtools/gcc/gcc-4.5.1.inc
@@ -57,6 +57,7 @@ SRC_URI = "${GNU_MIRROR}/gcc/gcc-${PV}/gcc-${PV}.tar.bz2 \
 	   file://gcc-poison-parameters.patch \
 	   file://GPLUSPLUS_INCLUDE_DIR_with_sysroot.patch \
 	   file://COLLECT_GCC_OPTIONS.patch \
+	   file://use-defaults.h-and-t-oe-in-B.patch \
 	  "
 	
 SRC_URI_append_sh3  = " file://sh3-installfix-fixheaders.patch;patch=1 "
diff --git a/meta/recipes-devtools/gcc/gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch b/meta/recipes-devtools/gcc/gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch
new file mode 100644
index 0000000..c93e6ca
--- /dev/null
+++ b/meta/recipes-devtools/gcc/gcc-4.5.1/use-defaults.h-and-t-oe-in-B.patch
@@ -0,0 +1,57 @@
+Upstream-Status: Pending
+
+Use the defaults.h in ${B} instead of ${S}, and t-oe in ${B}, so that
+the source can be shared between gcc-cross-initial,
+gcc-cross-intermediate, gcc-cross, gcc-runtime, and also the sdk build.
+---
+ gcc/Makefile.in  |    2 +-
+ gcc/configure    |    4 ++--
+ gcc/configure.ac |    4 ++--
+ 3 files changed, 5 insertions(+), 5 deletions(-)
+
+diff --git a/gcc/Makefile.in b/gcc/Makefile.in
+index d91f93a..03ee2bd 100644
+--- a/gcc/Makefile.in
++++ b/gcc/Makefile.in
+@@ -461,7 +461,7 @@ LIMITS_H_TEST = [ -f $(SYSTEM_HEADER_DIR)/limits.h ]
+ TARGET_SYSTEM_ROOT = @TARGET_SYSTEM_ROOT@
+ 
+ xmake_file=@xmake_file@
+-tmake_file=@tmake_file@
++tmake_file=@tmake_file@ ./t-oe
+ TM_ENDIAN_CONFIG=@TM_ENDIAN_CONFIG@
+ TM_MULTILIB_CONFIG=@TM_MULTILIB_CONFIG@
+ TM_MULTILIB_EXCEPTIONS_CONFIG=@TM_MULTILIB_EXCEPTIONS_CONFIG@
+diff --git a/gcc/configure b/gcc/configure
+index f440fa2..dafb0c1 100755
+--- a/gcc/configure
++++ b/gcc/configure
+@@ -10838,8 +10838,8 @@ for f in $tm_file; do
+        tm_include_list="${tm_include_list} $f"
+        ;;
+     defaults.h )
+-       tm_file_list="${tm_file_list} \$(srcdir)/$f"
+-       tm_include_list="${tm_include_list} $f"
++       tm_file_list="${tm_file_list} ./$f"
++       tm_include_list="${tm_include_list} ./$f"
+        ;;
+     * )
+        tm_file_list="${tm_file_list} \$(srcdir)/config/$f"
+diff --git a/gcc/configure.ac b/gcc/configure.ac
+index d003091..ba422e6 100644
+--- a/gcc/configure.ac
++++ b/gcc/configure.ac
+@@ -1652,8 +1652,8 @@ for f in $tm_file; do
+        tm_include_list="${tm_include_list} $f"
+        ;;
+     defaults.h )
+-       tm_file_list="${tm_file_list} \$(srcdir)/$f"
+-       tm_include_list="${tm_include_list} $f"
++       tm_file_list="${tm_file_list} ./$f"
++       tm_include_list="${tm_include_list} ./$f"
+        ;;
+     * )
+        tm_file_list="${tm_file_list} \$(srcdir)/config/$f"
+-- 
+1.7.1
+
-- 
1.7.1

[PATCH 4/6 V3] gcc-4.6.0: share work directories

[Robert Yang]

Fix configure and Makefile to read the defaults.h and t-oe from ${B},
so that the ${S} can be shared.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-4.6.0.inc            |    1 +
 .../gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch   |   80 ++++++++++++++++++++
 2 files changed, 81 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-devtools/gcc/gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch

diff --git a/meta/recipes-devtools/gcc/gcc-4.6.0.inc b/meta/recipes-devtools/gcc/gcc-4.6.0.inc
index cb9eb8e..ebfedd7 100644
--- a/meta/recipes-devtools/gcc/gcc-4.6.0.inc
+++ b/meta/recipes-devtools/gcc/gcc-4.6.0.inc
@@ -54,6 +54,7 @@ SRC_URI = "${GNU_MIRROR}/gcc/gcc-${PV}/gcc-${PV}.tar.bz2 \
 	   file://GPLUSPLUS_INCLUDE_DIR_with_sysroot.patch \
 	   file://COLLECT_GCC_OPTIONS.patch \
            file://volatile_access_backport.patch \
+           file://use-defaults.h-and-t-oe-in-B.patch \
 	  "
 
 SRC_URI_append_sh3  = " file://sh3-installfix-fixheaders.patch;patch=1 "
diff --git a/meta/recipes-devtools/gcc/gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch b/meta/recipes-devtools/gcc/gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch
new file mode 100644
index 0000000..b4351ee
--- /dev/null
+++ b/meta/recipes-devtools/gcc/gcc-4.6.0/use-defaults.h-and-t-oe-in-B.patch
@@ -0,0 +1,80 @@
+Upstream-Status: Pending
+
+Use the defaults.h in ${B} instead of ${S}, and t-oe in ${B}, so that
+the source can be shared between gcc-cross-initial,
+gcc-cross-intermediate, gcc-cross, gcc-runtime, and also the sdk build.
+---
+ gcc/Makefile.in  |    2 +-
+ gcc/configure    |    4 ++--
+ gcc/configure.ac |    4 ++--
+ gcc/mkconfig.sh  |    4 ++--
+ 4 files changed, 7 insertions(+), 7 deletions(-)
+
+diff --git a/gcc/Makefile.in b/gcc/Makefile.in
+index 7790915..3a0c34a 100644
+--- a/gcc/Makefile.in
++++ b/gcc/Makefile.in
+@@ -463,7 +463,7 @@ LIMITS_H_TEST = [ -f $(SYSTEM_HEADER_DIR)/limits.h ]
+ TARGET_SYSTEM_ROOT = @TARGET_SYSTEM_ROOT@
+ 
+ xmake_file=@xmake_file@
+-tmake_file=@tmake_file@
++tmake_file=@tmake_file@ ./t-oe
+ TM_ENDIAN_CONFIG=@TM_ENDIAN_CONFIG@
+ TM_MULTILIB_CONFIG=@TM_MULTILIB_CONFIG@
+ TM_MULTILIB_EXCEPTIONS_CONFIG=@TM_MULTILIB_EXCEPTIONS_CONFIG@
+diff --git a/gcc/configure b/gcc/configure
+index 82fa3e4..d4711b5 100755
+--- a/gcc/configure
++++ b/gcc/configure
+@@ -11227,8 +11227,8 @@ for f in $tm_file; do
+        tm_include_list="${tm_include_list} $f"
+        ;;
+     defaults.h )
+-       tm_file_list="${tm_file_list} \$(srcdir)/$f"
+-       tm_include_list="${tm_include_list} $f"
++       tm_file_list="${tm_file_list} ./$f"
++       tm_include_list="${tm_include_list} ./$f"
+        ;;
+     * )
+        tm_file_list="${tm_file_list} \$(srcdir)/config/$f"
+diff --git a/gcc/configure.ac b/gcc/configure.ac
+index 844d8da..a960343 100644
+--- a/gcc/configure.ac
++++ b/gcc/configure.ac
+@@ -1628,8 +1628,8 @@ for f in $tm_file; do
+        tm_include_list="${tm_include_list} $f"
+        ;;
+     defaults.h )
+-       tm_file_list="${tm_file_list} \$(srcdir)/$f"
+-       tm_include_list="${tm_include_list} $f"
++       tm_file_list="${tm_file_list} ./$f"
++       tm_include_list="${tm_include_list} ./$f"
+        ;;
+     * )
+        tm_file_list="${tm_file_list} \$(srcdir)/config/$f"
+diff --git a/gcc/mkconfig.sh b/gcc/mkconfig.sh
+index d56df8c..875d0f1 100644
+--- a/gcc/mkconfig.sh
++++ b/gcc/mkconfig.sh
+@@ -77,7 +77,7 @@ if [ -n "$HEADERS" ]; then
+     if [ $# -ge 1 ]; then
+ 	echo '#ifdef IN_GCC' >> ${output}T
+ 	for file in "$@"; do
+-	    if test x"$file" = x"defaults.h"; then
++	    if test x"$file" = x"./defaults.h"; then
+ 		postpone_defaults_h="yes"
+ 	    else
+ 		echo "# include \"$file\"" >> ${output}T
+@@ -103,7 +103,7 @@ esac
+ 
+ # If we postponed including defaults.h, add the #include now.
+ if test x"$postpone_defaults_h" = x"yes"; then
+-    echo "# include \"defaults.h\"" >> ${output}T
++    echo "# include \"./defaults.h\"" >> ${output}T
+ fi
+ 
+ # Add multiple inclusion protection guard, part two.
+-- 
+1.7.1
+
-- 
1.7.1

[PATCH 5/6 V3] gcc-crosssdk: share work directories

[Robert Yang]

The gcc-crosssdk.inc edited the files of config/*/linux*.h in ${S},
which made the source incompatible. Copy config to config-sdk, and edit
the files in config-sdk, configure will read the files in it when build
gcc-crosssdk.

The sed command for configure is more complicated than for configure.ac,
this is because there is such a line /boot/home/config/ in it, we should
not substitute it.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-crosssdk.inc |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-crosssdk.inc b/meta/recipes-devtools/gcc/gcc-crosssdk.inc
index 6e7d5a7..fdba6df 100644
--- a/meta/recipes-devtools/gcc/gcc-crosssdk.inc
+++ b/meta/recipes-devtools/gcc/gcc-crosssdk.inc
@@ -9,8 +9,17 @@ GCCMULTILIB = "--disable-multilib"
 DEPENDS = "virtual/${TARGET_PREFIX}binutils-crosssdk virtual/${TARGET_PREFIX}libc-for-gcc-nativesdk gettext-native"
 PROVIDES = "virtual/${TARGET_PREFIX}gcc-crosssdk virtual/${TARGET_PREFIX}g++-crosssdk"
 
+export BUILD_GCC_SDK = "-sdk"
 do_configure_prepend () {
 	# Change the default dynamic linker path to the one in the SDK
-	sed -i ${S}/gcc/config/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib/#\1${SYSTEMLIBS}#'
-	sed -i ${S}/gcc/config/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib64/#\1${SYSTEMLIBS}#'
+	[ ! -d ${S}/gcc/config-sdk ] && cp -r ${S}/gcc/config ${S}/gcc/config-sdk/
+	sed -i ${S}/gcc/configure.ac -e 's#config/#config\$BUILD_GCC_SDK/#'
+	sed -i ${S}/gcc/configure -e 's# config/# config\$BUILD_GCC_SDK/#' \
+		-e 's#\${srcdir}/config/#\${srcdir}/config\$BUILD_GCC_SDK/#' \
+		-e 's#\$srcdir/config/#\$srcdir/config\$BUILD_GCC_SDK/#' \
+		-e 's#\$(srcdir)/config/#\$(srcdir)/config\$BUILD_GCC_SDK/#' \
+		-e "s#\$(srcdir)'/config/#\$(srcdir)'/config\$BUILD_GCC_SDK/#"
+
+	sed -i ${S}/gcc/config-sdk/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib/#\1${SYSTEMLIBS}#'
+	sed -i ${S}/gcc/config-sdk/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib64/#\1${SYSTEMLIBS}#'
 }
-- 
1.7.1

Re: [PATCH 5/6 V3] gcc-crosssdk: share work directories

[Richard Purdie]

On Sat, 2011-06-18 at 20:41 +0800, Robert Yang wrote:
> The gcc-crosssdk.inc edited the files of config/*/linux*.h in ${S},
> which made the source incompatible. Copy config to config-sdk, and edit
> the files in config-sdk, configure will read the files in it when build
> gcc-crosssdk.
> 
> The sed command for configure is more complicated than for configure.ac,
> this is because there is such a line /boot/home/config/ in it, we should
> not substitute it.
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/recipes-devtools/gcc/gcc-crosssdk.inc |   13 +++++++++++--
>  1 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-devtools/gcc/gcc-crosssdk.inc b/meta/recipes-devtools/gcc/gcc-crosssdk.inc
> index 6e7d5a7..fdba6df 100644
> --- a/meta/recipes-devtools/gcc/gcc-crosssdk.inc
> +++ b/meta/recipes-devtools/gcc/gcc-crosssdk.inc
> @@ -9,8 +9,17 @@ GCCMULTILIB = "--disable-multilib"
>  DEPENDS = "virtual/${TARGET_PREFIX}binutils-crosssdk virtual/${TARGET_PREFIX}libc-for-gcc-nativesdk gettext-native"
>  PROVIDES = "virtual/${TARGET_PREFIX}gcc-crosssdk virtual/${TARGET_PREFIX}g++-crosssdk"
>  
> +export BUILD_GCC_SDK = "-sdk"
>  do_configure_prepend () {
>  	# Change the default dynamic linker path to the one in the SDK
> -	sed -i ${S}/gcc/config/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib/#\1${SYSTEMLIBS}#'
> -	sed -i ${S}/gcc/config/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib64/#\1${SYSTEMLIBS}#'
> +	[ ! -d ${S}/gcc/config-sdk ] && cp -r ${S}/gcc/config ${S}/gcc/config-sdk/
> +	sed -i ${S}/gcc/configure.ac -e 's#config/#config\$BUILD_GCC_SDK/#'
> +	sed -i ${S}/gcc/configure -e 's# config/# config\$BUILD_GCC_SDK/#' \
> +		-e 's#\${srcdir}/config/#\${srcdir}/config\$BUILD_GCC_SDK/#' \
> +		-e 's#\$srcdir/config/#\$srcdir/config\$BUILD_GCC_SDK/#' \
> +		-e 's#\$(srcdir)/config/#\$(srcdir)/config\$BUILD_GCC_SDK/#' \
> +		-e "s#\$(srcdir)'/config/#\$(srcdir)'/config\$BUILD_GCC_SDK/#"
> +
> +	sed -i ${S}/gcc/config-sdk/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib/#\1${SYSTEMLIBS}#'
> +	sed -i ${S}/gcc/config-sdk/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib64/#\1${SYSTEMLIBS}#'
>  }

The other patches look good but this one is still giving me cause for
concern I'm afraid. The reason why is that you could change SDKMACHINE
and get unexpected results. We really need the changes to be
configurable from ${B}.

How about this approach:

We always sed these files in config/ changing:

"GLIBC_DYNAMIC_LINKER.*/lib32/.*" -> "GLIBC_DYNAMIC_LINKER[...] (GLIBC_LIBDIR32 "/[...])"
"GLIBC_DYNAMIC_LINKER.*/lib64/.*" -> "GLIBC_DYNAMIC_LINKER[...] (GLIBC_LIBDIR64 "/[...])"
"GLIBC_DYNAMIC_LINKER.*/lib/.*" -> "GLIBC_DYNAMIC_LINKER[...] (GLIBC_LIBDIR "/[...])"

so entries in the files that look like:

#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"

would become

#define GLIBC_DYNAMIC_LINKER64 (GLIBC_LIBDIR64 "/ld-linux-x86-64.so.2")

and then we'd add:

#define GLIBC_LIBDIR "/lib"
#define GLIBC_LIBDIR32 "/lib32"
#define GLIBC_LIBDIR64 "/lib64"

to our config.h or t-oe (I haven't checked which which makes sense).
We'd then be able to customise these entries in the specific gcc
configurations as needed. It shouldn't change the "normal" cases and
when we get into multilib configurations, we should have some useful
code already present to assist with making filesystem layout changes too
as an added bonus.

Could you see if you can get something like that to work?

Cheers,

Richard

Re: [PATCH 5/6 V3] gcc-crosssdk: share work directories

[Robert Yang]

On 06/23/2011 02:55 AM, Richard Purdie wrote:
> On Sat, 2011-06-18 at 20:41 +0800, Robert Yang wrote:
>> The gcc-crosssdk.inc edited the files of config/*/linux*.h in ${S},
>> which made the source incompatible. Copy config to config-sdk, and edit
>> the files in config-sdk, configure will read the files in it when build
>> gcc-crosssdk.
>>
>> The sed command for configure is more complicated than for configure.ac,
>> this is because there is such a line /boot/home/config/ in it, we should
>> not substitute it.
>>
>> Signed-off-by: Robert Yang<liezhi.yang@windriver.com>
>> ---
>>   meta/recipes-devtools/gcc/gcc-crosssdk.inc |   13 +++++++++++--
>>   1 files changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-devtools/gcc/gcc-crosssdk.inc b/meta/recipes-devtools/gcc/gcc-crosssdk.inc
>> index 6e7d5a7..fdba6df 100644
>> --- a/meta/recipes-devtools/gcc/gcc-crosssdk.inc
>> +++ b/meta/recipes-devtools/gcc/gcc-crosssdk.inc
>> @@ -9,8 +9,17 @@ GCCMULTILIB = "--disable-multilib"
>>   DEPENDS = "virtual/${TARGET_PREFIX}binutils-crosssdk virtual/${TARGET_PREFIX}libc-for-gcc-nativesdk gettext-native"
>>   PROVIDES = "virtual/${TARGET_PREFIX}gcc-crosssdk virtual/${TARGET_PREFIX}g++-crosssdk"
>>
>> +export BUILD_GCC_SDK = "-sdk"
>>   do_configure_prepend () {
>>   	# Change the default dynamic linker path to the one in the SDK
>> -	sed -i ${S}/gcc/config/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib/#\1${SYSTEMLIBS}#'
>> -	sed -i ${S}/gcc/config/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib64/#\1${SYSTEMLIBS}#'
>> +	[ ! -d ${S}/gcc/config-sdk ]&&  cp -r ${S}/gcc/config ${S}/gcc/config-sdk/
>> +	sed -i ${S}/gcc/configure.ac -e 's#config/#config\$BUILD_GCC_SDK/#'
>> +	sed -i ${S}/gcc/configure -e 's# config/# config\$BUILD_GCC_SDK/#' \
>> +		-e 's#\${srcdir}/config/#\${srcdir}/config\$BUILD_GCC_SDK/#' \
>> +		-e 's#\$srcdir/config/#\$srcdir/config\$BUILD_GCC_SDK/#' \
>> +		-e 's#\$(srcdir)/config/#\$(srcdir)/config\$BUILD_GCC_SDK/#' \
>> +		-e "s#\$(srcdir)'/config/#\$(srcdir)'/config\$BUILD_GCC_SDK/#"
>> +
>> +	sed -i ${S}/gcc/config-sdk/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib/#\1${SYSTEMLIBS}#'
>> +	sed -i ${S}/gcc/config-sdk/*/linux*.h -e 's#\(GLIBC_DYNAMIC_LINKER.*\)/lib64/#\1${SYSTEMLIBS}#'
>>   }
>
> The other patches look good but this one is still giving me cause for
> concern I'm afraid. The reason why is that you could change SDKMACHINE
> and get unexpected results. We really need the changes to be
> configurable from ${B}.
>
> How about this approach:
>

Thanks, this approach seems better, I will send an update sooner.

// Robert

> We always sed these files in config/ changing:
>
> "GLIBC_DYNAMIC_LINKER.*/lib32/.*" ->  "GLIBC_DYNAMIC_LINKER[...] (GLIBC_LIBDIR32 "/[...])"
> "GLIBC_DYNAMIC_LINKER.*/lib64/.*" ->  "GLIBC_DYNAMIC_LINKER[...] (GLIBC_LIBDIR64 "/[...])"
> "GLIBC_DYNAMIC_LINKER.*/lib/.*" ->  "GLIBC_DYNAMIC_LINKER[...] (GLIBC_LIBDIR "/[...])"
>
> so entries in the files that look like:
>
> #define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
>
> would become
>
> #define GLIBC_DYNAMIC_LINKER64 (GLIBC_LIBDIR64 "/ld-linux-x86-64.so.2")
>
> and then we'd add:
>
> #define GLIBC_LIBDIR "/lib"
> #define GLIBC_LIBDIR32 "/lib32"
> #define GLIBC_LIBDIR64 "/lib64"
>
> to our config.h or t-oe (I haven't checked which which makes sense).
> We'd then be able to customise these entries in the specific gcc
> configurations as needed. It shouldn't change the "normal" cases and
> when we get into multilib configurations, we should have some useful
> code already present to assist with making filesystem layout changes too
> as an added bonus.
>
> Could you see if you can get something like that to work?
>
> Cheers,
>
> Richard
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

[PATCH 6/6 V3] gcc-cross-canadian: share work directories

[Robert Yang]

We had made gcc' configure to use the hacked defaults.h and t-oe in
${B}/gcc for gcc-cross, gcc-cross-canadian doesn't need the hacked
defatuls.h or t-oe, but we need make their source compatible, so:

1) copy ${S}/gcc/deafults.h(which is the original one with no change) to
   ${B}/gcc, it is the same as use the one in ${S}/gcc.

2) gcc-cross-canadian doesn't need the t-oe, so touch a empty t-oe for
   it, this doesn't change any behaviour of gcc-cross-canadian.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-configure-sdk.inc |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-configure-sdk.inc b/meta/recipes-devtools/gcc/gcc-configure-sdk.inc
index de80870..bf6fd8f 100644
--- a/meta/recipes-devtools/gcc/gcc-configure-sdk.inc
+++ b/meta/recipes-devtools/gcc/gcc-configure-sdk.inc
@@ -40,6 +40,14 @@ do_configure () {
 	export CXXFLAGS_FOR_BUILD="${BUILD_CXXFLAGS}"
 	export LDFLAGS_FOR_BUILD="${BUILD_LDFLAGS}"
 	(cd ${S} && gnu-configize) || die "failure running gnu-configize"
+
+	# This is to make the source of gcc and gcc-sdk compatible,
+	# the Makefile and configure of gcc need t-oe and defaults.h
+	# in ${B}, so fake two for gcc-sdk in ${B}
+	mkdir -p ${B}/gcc
+	touch ${B}/gcc/t-oe
+	cp ${S}/gcc/defaults.h ${B}/gcc/defaults.h
+
 	oe_runconf
 }
 
-- 
1.7.1

Re: [PATCH 0/6 V3] Share gcc work directories

[Koen Kooi]

Op 18 jun 2011, om 14:41 heeft Robert Yang het volgende geschreven:

> Changes of V3:
> 
> 1) Use --with-headers to replace the:
> 
>   sed -i 's:^\([ ]*\)glibc_header_dir=\"${with_build_sysroot}/usr/include\": ...
> 
> 2) Fix the build of gcc-cross-canadian
> 
> 3) Fix gcc-crosssdk which make the source incompatible during its
>   building.
> 
> 4) Test "bitbake meta-toolchain", and also do run test for the
>   pokysdk(unpack the toolchain to /opt, setup the env by sourcing
>   environment-setup-armv5te-poky-linux-gnueabi, and build some pkgs
>   out of poky, then use command file to check the binary file type.
> 
> 5) Make sure that no gcc build would make the source incompatible,
>   except gcc-cross_csl-arm-2008q1.bb, but I think this is fine, its
>   BINV = "4.2.3", not 4.5.1 or 4.6.0.


How does this interact with rm_work?

regards,

Koen

Re: [PATCH 0/6 V3] Share gcc work directories

[Robert Yang]

On 06/18/2011 08:50 PM, Koen Kooi wrote:
>
> Op 18 jun 2011, om 14:41 heeft Robert Yang het volgende geschreven:
>
>> Changes of V3:
>>
>> 1) Use --with-headers to replace the:
>>
>>    sed -i 's:^\([ ]*\)glibc_header_dir=\"${with_build_sysroot}/usr/include\": ...
>>
>> 2) Fix the build of gcc-cross-canadian
>>
>> 3) Fix gcc-crosssdk which make the source incompatible during its
>>    building.
>>
>> 4) Test "bitbake meta-toolchain", and also do run test for the
>>    pokysdk(unpack the toolchain to /opt, setup the env by sourcing
>>    environment-setup-armv5te-poky-linux-gnueabi, and build some pkgs
>>    out of poky, then use command file to check the binary file type.
>>
>> 5) Make sure that no gcc build would make the source incompatible,
>>    except gcc-cross_csl-arm-2008q1.bb, but I think this is fine, its
>>    BINV = "4.2.3", not 4.5.1 or 4.6.0.
>
>
> How does this interact with rm_work?
>

The rm_work does not delete anything in work-shared, I think this is fine,
otherwise the source can not be shared.

// Robert

> regards,
>
> Koen
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: [PATCH 0/6 V3] Share gcc work directories

[Khem Raj]

On Sun, Jun 19, 2011 at 6:48 PM, Robert Yang <liezhi.yang@windriver.com> wrote:
>> How does this interact with rm_work?
>>
>
> The rm_work does not delete anything in work-shared, I think this is fine,
> otherwise the source can not be shared.
>

it also limits the possibility of patching the sources differently if need be.
Can it do that easily suppose gcc intermediate needs a patch for a given arch
and others don't then can it make gcc cross intermediate not use shared source

Re: [PATCH 0/6 V3] Share gcc work directories

[Robert Yang]

On 06/21/2011 05:01 AM, Khem Raj wrote:
> On Sun, Jun 19, 2011 at 6:48 PM, Robert Yang<liezhi.yang@windriver.com>  wrote:
>>> How does this interact with rm_work?
>>>
>>
>> The rm_work does not delete anything in work-shared, I think this is fine,
>> otherwise the source can not be shared.
>>
>
> it also limits the possibility of patching the sources differently if need be.
> Can it do that easily suppose gcc intermediate needs a patch for a given arch


Yes, the source must be the same if they want to use the shared source. When we
want to patch the source, I think proper way is try to patch them general(not
only for a given arch).

> and others don't then can it make gcc cross intermediate not use shared source

Yes, when the following variables are not defined(or defined to other value)
for gcc intermediate, then it would not use the shared source.


S = "${TMPDIR}/work-shared/gcc-${PV}/gcc-${PV}"
B = "${WORKDIR}/gcc-${PV}/build.${HOST_SYS}.${TARGET_SYS}"

# SS means Shared Stamps directory
SS = "${TMPDIR}/stamps/work-shared/gcc-${PV}"
do_fetch[stamp-base] = "${SS}"
do_unpack[stamp-base] = "${SS}"
do_patch[stamp-base] = "${SS}"

# SW means Shared Work directory
SW = "${TMPDIR}/work-shared/gcc-${PV}"
WORKDIR_task-unpack = "${SW}"
WORKDIR_task-patch = "${SW}"

// Robert

>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>