public inbox for u-boot@lists.denx.de
 help / color / mirror / Atom feed
* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
@ 2012-04-25  5:33 Andy Fleming
  2012-04-25  8:22 ` Albert ARIBAUD
  2012-04-30 14:53 ` Wolfgang Denk
  0 siblings, 2 replies; 10+ messages in thread
From: Andy Fleming @ 2012-04-25  5:33 UTC (permalink / raw)
  To: u-boot

The MAKEALL script cleverly runs make with the appropriate options
to use all of the cores on the system, but your average U-Boot build
can't make much use of more than a few cores.  If you happen to have
a many-core server, your builds will leave most of the system idle.

In order to make full use of such a system, we need to build multiple
targets in parallel, and this requires directing make output into
multiple directories. We add a BUILD_NBUILDS variable, which allows
users to specify how many builds to run in parallel.
When BUILD_NBUILDS is set greater than 1, we redefine BUILD_DIR for
each build to be ${BUILD_DIR}/${target}. Also, we make "./build" the
default BUILD_DIR when BUILD_NBUILDS is greater than 1.

MAKEALL now tracks which builds are still running, and when one
finishes, it starts a new build.

Once each build finishes, we run "make tidy" on its directory, to reduce
the footprint.

As a result, we are left with a build directory with all of the built
targets still there for use, which means anyone who wanted to use
MAKEALL as part of a test harness can now do so.

Signed-off-by: Andy Fleming <afleming@freescale.com>
---
v2:	- Update to keep BUILD_NBUILDS builds in flight, rather than batching
	- Clean up style things
	- Defer error output until build completion to make output *slightly*
	more readable

v3:	- Add BUILD_MANY variable to clarify logic
	- Added comment for done/skip prefixes, and renamed for clarity
	- Changed CURRENT_COUNT to CURRENT_CNT to match TOTAL_CNT
	- Put build wait logic into its own manage_build function
	- Used wildcards to delete build management files
	- Fixed a bug where error-less builds printed an error

Once again, my unscientific tests on my 24-thread box:

time ./MAKEALL 83xx
real    4m2.951s
user    26m6.534s
sys     4m0.070s

time ./MAKEALL 86xx
real    0m27.271s
user    3m10.706s
sys     0m27.123s

time BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 86xx

real    0m38.430s
user    2m11.601s
sys     0m18.610s

time BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 83xx

real    1m15.642s
user    15m2.125s
sys     2m25.818s

time BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 85xx

real    4m29.840s
user    80m30.641s
sys     10m49.643s


Looks like 86xx was affected by having only 5 boards:

time BUILD_NBUILDS=10 BUILD_NCPUS=4 ./MAKEALL 86xx
real    0m11.887s
user    2m22.299s
sys     0m20.894s


 MAKEALL |  137 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 files changed, 122 insertions(+), 15 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index e5da6f1..e6c801c 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -34,6 +34,7 @@ usage()
 	  CROSS_COMPILE    cross-compiler toolchain prefix (default: "")
 	  MAKEALL_LOGDIR   output all logs to here (default: ./LOG/)
 	  BUILD_DIR        output build directory (default: ./)
+	  BUILD_NBUILDS	   number of parallel targets (default: 1)
 
 	Examples:
 	  - build all Power Architecture boards:
@@ -178,11 +179,22 @@ else
 	LOG_DIR="LOG"
 fi
 
-if [ ! "${BUILD_DIR}" ] ; then
-	BUILD_DIR="."
+: ${BUILD_NBUILDS:=1}
+BUILD_MANY=0
+
+if [ "${BUILD_NBUILDS}" -gt 1 ] ; then
+	BUILD_MANY=1
+	: ${BUILD_DIR:=./build}
+	mkdir -p "${BUILD_DIR}/ERR"
+	find "${BUILD_DIR}/ERR/" -type f -exec rm -f {} +
 fi
 
-[ -d ${LOG_DIR} ] || mkdir ${LOG_DIR} || exit 1
+: ${BUILD_DIR:=.}
+
+OUTPUT_PREFIX="${BUILD_DIR}"
+
+[ -d ${LOG_DIR} ] || mkdir "${LOG_DIR}" || exit 1
+find "${LOG_DIR}/" -type f -exec rm -f {} +
 
 LIST=""
 
@@ -190,6 +202,8 @@ LIST=""
 ERR_CNT=0
 ERR_LIST=""
 TOTAL_CNT=0
+CURRENT_CNT=0
+OLDEST_IDX=1
 RC=0
 
 # Helper funcs for parsing boards.cfg
@@ -592,8 +606,26 @@ list_target() {
 	echo ""
 }
 
+# Each finished build will have a file called ${donep}${n},
+# where n is the index of the build. Each build
+# we've already noted as finished will have ${skipp}${n}.
+# The code managing the build process will use this information
+# to ensure that only BUILD_NBUILDS builds are in flight at once
+donep="${LOG_DIR}/._done_"
+skipp="${LOG_DIR}/._skip_"
+
 build_target() {
 	target=$1
+	build_idx=$2
+
+	if [ $BUILD_MANY == 1 ] ; then
+		output_dir="${OUTPUT_PREFIX}/${target}"
+		mkdir -p "${output_dir}"
+	else
+		output_dir="${OUTPUT_PREFIX}"
+	fi
+
+	export BUILD_DIR="${output_dir}"
 
 	if [ "$ONLY_LIST" == 'y' ] ; then
 		list_target ${target}
@@ -603,30 +635,75 @@ build_target() {
 	${MAKE} distclean >/dev/null
 	${MAKE} -s ${target}_config
 
-	${MAKE} ${JOBS} all 2>&1 >${LOG_DIR}/$target.MAKELOG \
-				| tee ${LOG_DIR}/$target.ERR
+	${MAKE} ${JOBS} all \
+		>${LOG_DIR}/$target.MAKELOG 2> ${LOG_DIR}/$target.ERR
 
 	# Check for 'make' errors
 	if [ ${PIPESTATUS[0]} -ne 0 ] ; then
 		RC=1
 	fi
 
-	if [ -s ${LOG_DIR}/$target.ERR ] ; then
-		ERR_CNT=$((ERR_CNT + 1))
-		ERR_LIST="${ERR_LIST} $target"
+	if [ $BUILD_MANY == 1 ] ; then
+		${MAKE} tidy
+
+		if [ -s ${LOG_DIR}/${target}.ERR ] ; then
+			touch ${OUTPUT_PREFIX}/ERR/${target}
+		else
+			rm ${LOG_DIR}/${target}.ERR
+		fi
 	else
-		rm ${LOG_DIR}/$target.ERR
+		if [ -s ${LOG_DIR}/${target}.ERR ] ; then
+			: $(( ERR_CNT += 1 ))
+			ERR_LIST="${ERR_LIST} $target"
+		else
+			rm ${LOG_DIR}/${target}.ERR
+		fi
 	fi
 
-	TOTAL_CNT=$((TOTAL_CNT + 1))
-
-	OBJS=${BUILD_DIR}/u-boot
-	if [ -e ${BUILD_DIR}/spl/u-boot-spl ]; then
-		OBJS="${OBJS} ${BUILD_DIR}/spl/u-boot-spl"
+	OBJS=${output_dir}/u-boot
+	if [ -e ${output_dir}/spl/u-boot-spl ]; then
+		OBJS="${OBJS} ${output_dir}/spl/u-boot-spl"
 	fi
 
 	${CROSS_COMPILE}size ${OBJS} | tee -a ${LOG_DIR}/$target.MAKELOG
+
+	[ -e "${LOG_DIR}/${target}.ERR" ] && cat "${LOG_DIR}/${target}.ERR"
+
+	#echo "Writing ${donep}${build_idx}"
+	touch "${donep}${build_idx}"
 }
+
+manage_builds() {
+	search_idx=${OLDEST_IDX}
+	#echo "Searching ${OLDEST_IDX} to ${TOTAL_CNT}"
+	while true; do
+		if [ -e "${donep}${search_idx}" ] ; then
+	#		echo "Found ${donep}${search_idx}"
+			: $(( CURRENT_CNT-- ))
+			[ ${OLDEST_IDX} -eq ${search_idx} ] &&
+				: $(( OLDEST_IDX++ ))
+
+			# Only want to count it once
+			rm -f "${donep}${search_idx}"
+			touch "${skipp}${search_idx}"
+		elif [ -e "${skipp}${search_idx}" ] ; then
+			[ ${OLDEST_IDX} -eq ${search_idx} ] &&
+				: $(( OLDEST_IDX++ ))
+		fi
+		#echo "Checking search ${search_idx} vs ${TOTAL_CNT}"
+		: $(( search_idx++ ))
+		if [ ${search_idx} -gt ${TOTAL_CNT} ] ; then
+			#echo "Checking current ${CURRENT_CNT} vs ${BUILD_NBUILDS}"
+			if [ ${CURRENT_CNT} -ge ${BUILD_NBUILDS} ] ; then
+				search_idx=${OLDEST_IDX}
+				sleep 1
+			else
+				break
+			fi
+		fi
+	done
+}
+
 build_targets() {
 	for t in "$@" ; do
 		# If a LIST_xxx var exists, use it.  But avoid variable
@@ -639,7 +716,26 @@ build_targets() {
 		if [ -n "${list}" ] ; then
 			build_targets ${list}
 		else
-			build_target ${t}
+			: $((TOTAL_CNT += 1))
+			: $((CURRENT_CNT += 1))
+			rm -f "${donep}${TOTAL_CNT}"
+			rm -f "${skipp}${TOTAL_CNT}"
+			build_target ${t} ${TOTAL_CNT} &
+		fi
+
+		# We maintain a running count of all the builds we have done.
+		# Each finished build will have a file called ${donep}${n},
+		# where n is the index of the build. Each build
+		# we've already noted as finished will have ${skipp}${n}.
+		# We track the current index via TOTAL_CNT, and the oldest
+		# index. When we exceed the maximum number of parallel builds,
+		# We look from oldest to current for builds that have completed,
+		# and update the current count and oldest index as appropriate.
+		# If we've gone through the entire list, wait a second, and
+		# reprocess the entire list until we find a build that has
+		# completed
+		if [ ${CURRENT_CNT} -ge ${BUILD_NBUILDS} ] ; then
+			manage_builds
 		fi
 	done
 }
@@ -648,6 +744,16 @@ build_targets() {
 
 print_stats() {
 	if [ "$ONLY_LIST" == 'y' ] ; then return ; fi
+
+	rm -f ${donep}* ${skipp}*
+
+	if [ $BUILD_MANY == 1 ] && [ -e "${OUTPUT_PREFIX}/ERR" ] ; then
+		ERR_LIST=$(ls ${OUTPUT_PREFIX}/ERR/)
+		ERR_CNT=`ls -1 ${OUTPUT_PREFIX}/ERR/ | wc | awk '{print $1}'`
+	else
+		ERR_CNT=0
+	fi
+
 	echo ""
 	echo "--------------------- SUMMARY ----------------------------"
 	echo "Boards compiled: ${TOTAL_CNT}"
@@ -666,3 +772,4 @@ set -- ${SELECTED} "$@"
 # run PowerPC by default
 [ $# = 0 ] && set -- powerpc
 build_targets "$@"
+wait
-- 
1.7.3.4

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-04-25  5:33 [U-Boot] [PATCH v3] Allow for parallel builds and saved output Andy Fleming
@ 2012-04-25  8:22 ` Albert ARIBAUD
  2012-04-25 22:07   ` Andy Fleming
  2012-04-30 14:53 ` Wolfgang Denk
  1 sibling, 1 reply; 10+ messages in thread
From: Albert ARIBAUD @ 2012-04-25  8:22 UTC (permalink / raw)
  To: u-boot

Hi Andy,

Le 25/04/2012 07:33, Andy Fleming a ?crit :
> The MAKEALL script cleverly runs make with the appropriate options
> to use all of the cores on the system, but your average U-Boot build
> can't make much use of more than a few cores.  If you happen to have
> a many-core server, your builds will leave most of the system idle.
>
> In order to make full use of such a system, we need to build multiple
> targets in parallel, and this requires directing make output into
> multiple directories. We add a BUILD_NBUILDS variable, which allows
> users to specify how many builds to run in parallel.
> When BUILD_NBUILDS is set greater than 1, we redefine BUILD_DIR for
> each build to be ${BUILD_DIR}/${target}. Also, we make "./build" the
> default BUILD_DIR when BUILD_NBUILDS is greater than 1.
>
> MAKEALL now tracks which builds are still running, and when one
> finishes, it starts a new build.
>
> Once each build finishes, we run "make tidy" on its directory, to reduce
> the footprint.
>
> As a result, we are left with a build directory with all of the built
> targets still there for use, which means anyone who wanted to use
> MAKEALL as part of a test harness can now do so.
>
> Signed-off-by: Andy Fleming<afleming@freescale.com>
> ---
> v2:	- Update to keep BUILD_NBUILDS builds in flight, rather than batching
> 	- Clean up style things
> 	- Defer error output until build completion to make output *slightly*
> 	more readable
>
> v3:	- Add BUILD_MANY variable to clarify logic
> 	- Added comment for done/skip prefixes, and renamed for clarity
> 	- Changed CURRENT_COUNT to CURRENT_CNT to match TOTAL_CNT
> 	- Put build wait logic into its own manage_build function
> 	- Used wildcards to delete build management files
> 	- Fixed a bug where error-less builds printed an error

For ./MAKEALL arm, using cs2011-09, 7 builds 1 cpu on a 8 thread machine:

real	29m20.272s
user	95m59.708s
sys	10m17.383s

without the patch, and

real	20m26.480s
user	100m1.155s
sys	8m34.988s

with it, so there is definitely a benefit.

*However*, two boards build without and do not build clean with: scpu 
and pndb3.

Also, the final report is not displayed as the same with and without the 
patch -- with it, unclean boards are listed one per line, without it, 
they are listed in a single line.

Last: breaking during a parallel build then trying a git clean -xfd 
causes errors such as files missing.

Amicalement,
-- 
Albert.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-04-25  8:22 ` Albert ARIBAUD
@ 2012-04-25 22:07   ` Andy Fleming
  0 siblings, 0 replies; 10+ messages in thread
From: Andy Fleming @ 2012-04-25 22:07 UTC (permalink / raw)
  To: u-boot

On Wed, Apr 25, 2012 at 3:22 AM, Albert ARIBAUD
<albert.u.boot@aribaud.net> wrote:
> Hi Andy,
>
> Le 25/04/2012 07:33, Andy Fleming a ?crit :

> *However*, two boards build without and do not build clean with: scpu and
> pndb3.

Ok, I figured that one out, and sent out a patch.


>
> Also, the final report is not displayed as the same with and without the
> patch -- with it, unclean boards are listed one per line, without it, they
> are listed in a single line.

I can probably fix that, no problem.


>
> Last: breaking during a parallel build then trying a git clean -xfd causes
> errors such as files missing.


I believe what you're seeing is that there are still builds running
after you've hit ctrl-C, and they continue until they're done. I think
I've got it figured out, so I'll fix that in my followup.

Andy

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-04-25  5:33 [U-Boot] [PATCH v3] Allow for parallel builds and saved output Andy Fleming
  2012-04-25  8:22 ` Albert ARIBAUD
@ 2012-04-30 14:53 ` Wolfgang Denk
  2012-05-06  1:48   ` Marek Vasut
  1 sibling, 1 reply; 10+ messages in thread
From: Wolfgang Denk @ 2012-04-30 14:53 UTC (permalink / raw)
  To: u-boot

Dear Andy Fleming,

In message <1335332031-24138-1-git-send-email-afleming@freescale.com> you wrote:
> The MAKEALL script cleverly runs make with the appropriate options
> to use all of the cores on the system, but your average U-Boot build
> can't make much use of more than a few cores.  If you happen to have
> a many-core server, your builds will leave most of the system idle.
> 
> In order to make full use of such a system, we need to build multiple
> targets in parallel, and this requires directing make output into
> multiple directories. We add a BUILD_NBUILDS variable, which allows
> users to specify how many builds to run in parallel.
> When BUILD_NBUILDS is set greater than 1, we redefine BUILD_DIR for
> each build to be ${BUILD_DIR}/${target}. Also, we make "./build" the
> default BUILD_DIR when BUILD_NBUILDS is greater than 1.
> 
> MAKEALL now tracks which builds are still running, and when one
> finishes, it starts a new build.
> 
> Once each build finishes, we run "make tidy" on its directory, to reduce
> the footprint.
> 
> As a result, we are left with a build directory with all of the built
> targets still there for use, which means anyone who wanted to use
> MAKEALL as part of a test harness can now do so.
> 
> Signed-off-by: Andy Fleming <afleming@freescale.com>
> ---
> v2:	- Update to keep BUILD_NBUILDS builds in flight, rather than batching
> 	- Clean up style things
> 	- Defer error output until build completion to make output *slightly*
> 	more readable
> 
> v3:	- Add BUILD_MANY variable to clarify logic
> 	- Added comment for done/skip prefixes, and renamed for clarity
> 	- Changed CURRENT_COUNT to CURRENT_CNT to match TOTAL_CNT
> 	- Put build wait logic into its own manage_build function
> 	- Used wildcards to delete build management files
> 	- Fixed a bug where error-less builds printed an error
> 
> Once again, my unscientific tests on my 24-thread box:
> 
> time ./MAKEALL 83xx
> real    4m2.951s
> user    26m6.534s
> sys     4m0.070s
> 
> time ./MAKEALL 86xx
> real    0m27.271s
> user    3m10.706s
> sys     0m27.123s
> 
> time BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 86xx
> 
> real    0m38.430s
> user    2m11.601s
> sys     0m18.610s
> 
> time BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 83xx
> 
> real    1m15.642s
> user    15m2.125s
> sys     2m25.818s
> 
> time BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 85xx
> 
> real    4m29.840s
> user    80m30.641s
> sys     10m49.643s
> 
> 
> Looks like 86xx was affected by having only 5 boards:
> 
> time BUILD_NBUILDS=10 BUILD_NCPUS=4 ./MAKEALL 86xx
> real    0m11.887s
> user    2m22.299s
> sys     0m20.894s
> 
> 
>  MAKEALL |  137 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
>  1 files changed, 122 insertions(+), 15 deletions(-)

Applied, thanks.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
We Klingons believe as you do -- the sick should die. Only the strong
should live.
	-- Kras, "Friday's Child", stardate 3497.2

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-04-30 14:53 ` Wolfgang Denk
@ 2012-05-06  1:48   ` Marek Vasut
  2012-05-08 19:17     ` Albert ARIBAUD
  0 siblings, 1 reply; 10+ messages in thread
From: Marek Vasut @ 2012-05-06  1:48 UTC (permalink / raw)
  To: u-boot

Dear Wolfgang Denk,

> Dear Andy Fleming,
> 
> In message <1335332031-24138-1-git-send-email-afleming@freescale.com> you 
wrote:
> > The MAKEALL script cleverly runs make with the appropriate options
> > to use all of the cores on the system, but your average U-Boot build
> > can't make much use of more than a few cores.  If you happen to have
> > a many-core server, your builds will leave most of the system idle.
[...]
> >  MAKEALL |  137
> >  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 files
> >  changed, 122 insertions(+), 15 deletions(-)
> 
> Applied, thanks.

What do you get if you run:

./MAKEALL -a arm -l

? :-)

> 
> Best regards,
> 
> Wolfgang Denk

Best regards,
Marek Vasut

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-05-06  1:48   ` Marek Vasut
@ 2012-05-08 19:17     ` Albert ARIBAUD
  2012-05-08 19:19       ` Andy Fleming
  0 siblings, 1 reply; 10+ messages in thread
From: Albert ARIBAUD @ 2012-05-08 19:17 UTC (permalink / raw)
  To: u-boot

Le 06/05/2012 03:48, Marek Vasut a ?crit :
> Dear Wolfgang Denk,
>
>> Dear Andy Fleming,
>>
>> In message<1335332031-24138-1-git-send-email-afleming@freescale.com>  you
> wrote:
>>> The MAKEALL script cleverly runs make with the appropriate options
>>> to use all of the cores on the system, but your average U-Boot build
>>> can't make much use of more than a few cores.  If you happen to have
>>> a many-core server, your builds will leave most of the system idle.
> [...]
>>>   MAKEALL |  137
>>>   ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 files
>>>   changed, 122 insertions(+), 15 deletions(-)
>>
>> Applied, thanks.
>
> What do you get if you run:
>
> ./MAKEALL -a arm -l
>
> ? :-)

I am being hit by a spurious build failure in parallel build: pdnb3 
succeeds when built alone but fails in a MAKEALL arm with the following 
.ERR:

mv: cannot stat 
`/home/uboot/src/u-boot-arm/build/pdnb3/include/autoconf.mk.tmp': No 
such file or directory

Looks like several builds competed for this temp file name :)...

Amicalement,
-- 
Albert.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-05-08 19:17     ` Albert ARIBAUD
@ 2012-05-08 19:19       ` Andy Fleming
  2012-05-08 19:50         ` Marek Vasut
  0 siblings, 1 reply; 10+ messages in thread
From: Andy Fleming @ 2012-05-08 19:19 UTC (permalink / raw)
  To: u-boot


On May 8, 2012, at 2:17 PM, Albert ARIBAUD wrote:

> Le 06/05/2012 03:48, Marek Vasut a ?crit :
>> Dear Wolfgang Denk,
>> 
>>> Dear Andy Fleming,
>>> 
>>> In message<1335332031-24138-1-git-send-email-afleming@freescale.com>  you
>> wrote:
>>>> The MAKEALL script cleverly runs make with the appropriate options
>>>> to use all of the cores on the system, but your average U-Boot build
>>>> can't make much use of more than a few cores.  If you happen to have
>>>> a many-core server, your builds will leave most of the system idle.
>> [...]
>>>>  MAKEALL |  137
>>>>  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 files
>>>>  changed, 122 insertions(+), 15 deletions(-)
>>> 
>>> Applied, thanks.
>> 
>> What do you get if you run:
>> 
>> ./MAKEALL -a arm -l
>> 
>> ? :-)
> 
> I am being hit by a spurious build failure in parallel build: pdnb3 succeeds when built alone but fails in a MAKEALL arm with the following .ERR:
> 
> mv: cannot stat `/home/uboot/src/u-boot-arm/build/pdnb3/include/autoconf.mk.tmp': No such file or directory
> 
> Looks like several builds competed for this temp file name :)...


I already submitted the patch for this. It's because pdnb3 was being built twice (and therefore twice concurrently). Wolfgang just has to apply that patch, too. :)

Andy

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-05-08 19:19       ` Andy Fleming
@ 2012-05-08 19:50         ` Marek Vasut
  2012-05-08 19:55           ` Andy Fleming
  2012-05-18 11:47           ` Wolfgang Denk
  0 siblings, 2 replies; 10+ messages in thread
From: Marek Vasut @ 2012-05-08 19:50 UTC (permalink / raw)
  To: u-boot

Dear Andy Fleming,

> On May 8, 2012, at 2:17 PM, Albert ARIBAUD wrote:
> > Le 06/05/2012 03:48, Marek Vasut a ?crit :
> >> Dear Wolfgang Denk,
> >> 
> >>> Dear Andy Fleming,
> >>> 
> >>> In message<1335332031-24138-1-git-send-email-afleming@freescale.com> 
> >>> you
> >> 
> >> wrote:
> >>>> The MAKEALL script cleverly runs make with the appropriate options
> >>>> to use all of the cores on the system, but your average U-Boot build
> >>>> can't make much use of more than a few cores.  If you happen to have
> >>>> a many-core server, your builds will leave most of the system idle.
> >> 
> >> [...]
> >> 
> >>>>  MAKEALL |  137
> >>>>  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1
> >>>>  files changed, 122 insertions(+), 15 deletions(-)
> >>> 
> >>> Applied, thanks.
> >> 
> >> What do you get if you run:
> >> 
> >> ./MAKEALL -a arm -l
> >> 
> >> ? :-)
> > 
> > I am being hit by a spurious build failure in parallel build: pdnb3
> > succeeds when built alone but fails in a MAKEALL arm with the following
> > .ERR:
> > 
> > mv: cannot stat
> > `/home/uboot/src/u-boot-arm/build/pdnb3/include/autoconf.mk.tmp': No
> > such file or directory
> > 
> > Looks like several builds competed for this temp file name :)...
> 
> I already submitted the patch for this. It's because pdnb3 was being built
> twice (and therefore twice concurrently). Wolfgang just has to apply that
> patch, too. :)

Does it also fix ./MAKEALL -a arm -l ?

> 
> Andy

Best regards,
Marek Vasut

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-05-08 19:50         ` Marek Vasut
@ 2012-05-08 19:55           ` Andy Fleming
  2012-05-18 11:47           ` Wolfgang Denk
  1 sibling, 0 replies; 10+ messages in thread
From: Andy Fleming @ 2012-05-08 19:55 UTC (permalink / raw)
  To: u-boot


On May 8, 2012, at 2:50 PM, Marek Vasut wrote:

> Dear Andy Fleming,
> 
>> On May 8, 2012, at 2:17 PM, Albert ARIBAUD wrote:
>>> Le 06/05/2012 03:48, Marek Vasut a ?crit :
>>>> Dear Wolfgang Denk,
>>>> 
>>>>> Dear Andy Fleming,
>>>>> 
>>>>> In message<1335332031-24138-1-git-send-email-afleming@freescale.com> 
>>>>> you
>>>> 
>>>> wrote:
>>>>>> The MAKEALL script cleverly runs make with the appropriate options
>>>>>> to use all of the cores on the system, but your average U-Boot build
>>>>>> can't make much use of more than a few cores.  If you happen to have
>>>>>> a many-core server, your builds will leave most of the system idle.
>>>> 
>>>> [...]
>>>> 
>>>>>> MAKEALL |  137
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1
>>>>>> files changed, 122 insertions(+), 15 deletions(-)
>>>>> 
>>>>> Applied, thanks.
>>>> 
>>>> What do you get if you run:
>>>> 
>>>> ./MAKEALL -a arm -l
>>>> 
>>>> ? :-)
>>> 
>>> I am being hit by a spurious build failure in parallel build: pdnb3
>>> succeeds when built alone but fails in a MAKEALL arm with the following
>>> .ERR:
>>> 
>>> mv: cannot stat
>>> `/home/uboot/src/u-boot-arm/build/pdnb3/include/autoconf.mk.tmp': No
>>> such file or directory
>>> 
>>> Looks like several builds competed for this temp file name :)...
>> 
>> I already submitted the patch for this. It's because pdnb3 was being built
>> twice (and therefore twice concurrently). Wolfgang just has to apply that
>> patch, too. :)
> 
> Does it also fix ./MAKEALL -a arm -l ?

Probably not. I was working on a follow-on patch that *probably* fixes that (I haven't had time to run your test). I'll submit that when I manage to get these MMC patches applied.

Andy

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [U-Boot] [PATCH v3] Allow for parallel builds and saved output
  2012-05-08 19:50         ` Marek Vasut
  2012-05-08 19:55           ` Andy Fleming
@ 2012-05-18 11:47           ` Wolfgang Denk
  1 sibling, 0 replies; 10+ messages in thread
From: Wolfgang Denk @ 2012-05-18 11:47 UTC (permalink / raw)
  To: u-boot

Dear Marek Vasut,

In message <201205082150.05943.marex@denx.de> you wrote:
> 
> > I already submitted the patch for this. It's because pdnb3 was being built
> > twice (and therefore twice concurrently). Wolfgang just has to apply that
> > patch, too. :)
> 
> Does it also fix ./MAKEALL -a arm -l ?

No, it doesn't this still hangs, even after applying Andy's patch.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Our management frequently gets lost in thought.   That's because it's
unfamiliar territory.

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2012-05-18 11:47 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-04-25  5:33 [U-Boot] [PATCH v3] Allow for parallel builds and saved output Andy Fleming
2012-04-25  8:22 ` Albert ARIBAUD
2012-04-25 22:07   ` Andy Fleming
2012-04-30 14:53 ` Wolfgang Denk
2012-05-06  1:48   ` Marek Vasut
2012-05-08 19:17     ` Albert ARIBAUD
2012-05-08 19:19       ` Andy Fleming
2012-05-08 19:50         ` Marek Vasut
2012-05-08 19:55           ` Andy Fleming
2012-05-18 11:47           ` Wolfgang Denk

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox