[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Simon Glass]

Buildman has been around for a little over a year and is used by a fair
number of U-Boot developers. However quite a few people still use MAKEALL.

Buildman was intended to replace MAKEALL, so perhaps now is a good time to
start that process.

The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more features

This series adds a few features to buildman to fill some gaps, adds some
infomation into the README on how to migrate from MAKEALL, and adds a
deprecation message to MAKEALL.

Changes in v2:
- Add new patch to fix existing typos
- Minor changes to the text

Simon Glass (3):
  buildman: Fix a few typos
  buildman: Add some notes about moving from MAKEALL
  RFC: Deprecate MAKEALL

 MAKEALL               |   4 ++
 tools/buildman/README | 104 +++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 102 insertions(+), 6 deletions(-)

-- 
2.0.0.526.g5318336

[U-Boot] [PATCH v2 1/3] buildman: Fix a few typos

[Simon Glass]

There are several typos in the README - fix them.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

Changes in v2:
- Add new patch to fix existing typos

 tools/buildman/README | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/buildman/README b/tools/buildman/README
index c30c1d4..a5d181c 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -71,8 +71,8 @@ directory. It clones this repository into a copy for each thread, and the
 threads do not affect the state of your git repository. Any checkouts done
 by the thread affect only the working directory for that thread.
 
-Buildman automatically selects the correct toolchain for each board. You
-must supply suitable toolchains, but buildman takes care of selecting the
+Buildman automatically selects the correct tool chain for each board. You
+must supply suitable tool chains, but buildman takes care of selecting the
 right one.
 
 Buildman always builds a branch, and always builds the upstream commit as
@@ -287,7 +287,7 @@ If it can't detect the upstream branch, try checking out the branch, and
 doing something like 'git branch --set-upstream <branch> upstream/master'
 or something similar.
 
-As an exmmple:
+As an example:
 
 Dry run, so not doing much. But I would do this:
 
@@ -339,7 +339,7 @@ Building 18 commits for 1059 boards (4 threads, 1 job per thread)
   528   36  124 /19062  1:13:30  : SIMPC8313_SP
 
 This means that it is building 19062 board/commit combinations. So far it
-has managed to succesfully build 528. Another 36 have built with warnings,
+has managed to successfully build 528. Another 36 have built with warnings,
 and 124 more didn't build at all. Buildman expects to complete the process
 in an hour and 15 minutes. Use this time to buy a faster computer.
 
@@ -413,7 +413,7 @@ again.
 
 At commit 16, the error moves - you can see that the old error at line 120
 is fixed, but there is a new one at line 126. This is probably only because
-we added some code and moved the broken line futher down the file.
+we added some code and moved the broken line father down the file.
 
 If many boards have the same error, then -e will display the error only
 once. This makes the output as concise as possible.
@@ -491,7 +491,7 @@ You can also use -d to see a detailed size breakdown for each board. This
 list is sorted in order from largest growth to largest reduction.
 
 It is possible to go a little further with the -B option (--bloat). This
-shows where U-Boot has bloted, breaking the size change down to the function
+shows where U-Boot has bloated, breaking the size change down to the function
 level. Example output is below:
 
 $ ./tools/buildman/buildman -b us-mem4 -sSdB
-- 
2.0.0.526.g5318336

[U-Boot] [PATCH v2 2/3] buildman: Add some notes about moving from MAKEALL

[Simon Glass]

For those used to MAKEALL, buildman seems strange. Add some notes to ease
the transition.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

Changes in v2:
- Minor changes to the text

 tools/buildman/README | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)

diff --git a/tools/buildman/README b/tools/buildman/README
index a5d181c..1c919af 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -3,6 +3,8 @@
 # SPDX-License-Identifier:	GPL-2.0+
 #
 
+(Please read 'How to change from MAKEALL' if you are used to that tool)
+
 What is this?
 =============
 
@@ -663,6 +665,96 @@ Other options
 Buildman has various other command line options. Try --help to see them.
 
 
+How to change from MAKEALL
+==========================
+
+Buildman includes most of the features of MAKEALL and is generally faster
+and easier to use. In particular it builds entire branches: if a particular
+commit introduces an error in a particular board, buildman can easily show
+you this, even if a later commit fixes that error.
+
+The reasons to deprecate MAKEALL are:
+- We don't want to maintain two build systems
+- Buildman is typically faster
+- Buildman has a lot more features
+
+But still, many people will be sad to lose MAKEALL. If you are used to
+MAKEALL, here are a few pointers.
+
+First you need to set up your tool chains - see the 'Setting up' section
+for details. Once you have your required toolchain(s) detected then you are
+ready to go.
+
+Buildman works on entire branches, so the normal use is:
+
+   ./tools/buildman/buildman -b <branch_name> <list of things to build>
+
+followed by (afterwards, or perhaps concurrently in another terminal):
+
+   ./tools/buildman/buildman -b <branch_name> -s <list of things to build>
+
+to see the results of the build. Rather than showing you all the output,
+buildman just shows a summary, with red indicating that a commit introduced
+an error and green indicating that a commit fixed an error. Use the -e
+flag to see the full errors.
+
+You don't need to stick around on that branch while buildman is running. It
+checks out its own copy of the source code, so you can change branches,
+add commits, etc. without affecting the build in progress.
+
+The <list of things to build> can include board names, architectures or the
+like. There are no flags to disambiguate since ambiguities are rare. Using
+the examples from MAKEALL:
+
+Examples:
+  - build all Power Architecture boards:
+      MAKEALL -a powerpc
+      MAKEALL --arch powerpc
+      MAKEALL powerpc
+          ** buildman -b <branch> powerpc
+  - build all PowerPC boards manufactured by vendor "esd":
+      MAKEALL -a powerpc -v esd
+          ** buildman -b <branch> esd
+  - build all PowerPC boards manufactured either by "keymile" or "siemens":
+      MAKEALL -a powerpc -v keymile -v siemens
+          ** buildman -b <branch> keymile siemens
+  - build all Freescale boards with MPC83xx CPUs, plus all 4xx boards:
+      MAKEALL -c mpc83xx -v freescale 4xx
+          ** buildman -b <branch> mpc83xx freescale 4xx
+
+Buildman automatically tries to use all the CPUs in your machine. If you
+are building a lot of boards it will use one thread for every CPU core
+it detects in your machine. This is like MAKEALL's BUILD_NBUILDS option.
+You can use the -T flag to change the number of threads. If you are only
+building a few boards, buildman will automatically run make with the -j
+flag to increase the number of concurrent make tasks. It isn't normally
+that helpful to fiddle with this option, but if you use the BUILD_NCPUS
+option in MAKEALL then -j is the equivalent in buildman.
+
+Buildman puts its output in ../<branch_name> by default but you can change
+this with the -o option. Buildman normally does out-of-tree builds: use -i
+to disable that if you really want to. But be careful that once you have
+used -i you pollute buildman's copies of the source tree, and you will need
+to remove the build directory (normally ../<branch_name>) to run buildman
+in normal mode (without -i).
+
+Buildman doesn't keep the output result normally, but use the -k option to
+do this.
+
+Please read 'Theory of Operation' a few times as it will make a lot of
+things clearer.
+
+Some options you might like are:
+
+   -B shows which functions are growing/shrinking in which commit - great
+        for finding code bloat.
+   -S shows image sizes for each commit (just an overall summary)
+   -u shows boards that you haven't built yet
+   --step 0 will build just the upstream commit and the last commit of your
+        branch. This is often a quick sanity check that your branch doesn't
+        break anything. But note this does not check bisectability!
+
+
 TODO
 ====
 
-- 
2.0.0.526.g5318336

[U-Boot] [PATCH v2 3/3] RFC: Deprecate MAKEALL

[Simon Glass]

Since buildman now includes most of the features of MAKEALL it is proabably
time to talk about deprecating MAKEALL.

Comments welcome.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

Changes in v2: None

 MAKEALL | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/MAKEALL b/MAKEALL
index 020e65f..6443814 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -60,6 +60,10 @@ usage()
 	exit ${ret}
 }
 
+echo "** Note: MAKEALL is deprecated - please use buildman instead"
+echo "** See tools/buildman/README for details"
+echo
+
 SHORT_OPTS="ha:c:v:s:b:lmMCnr"
 LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
 
-- 
2.0.0.526.g5318336

[U-Boot] [PATCH v2 2/3] buildman: Add some notes about moving from MAKEALL

[Masahiro Yamada]

Hi.


On Mon, 28 Jul 2014 23:53:29 -0600
Simon Glass <sjg@chromium.org> wrote:

> For those used to MAKEALL, buildman seems strange. Add some notes to ease
> the transition.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
> 
> Changes in v2:
> - Minor changes to the text
> 
>  tools/buildman/README | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 92 insertions(+)
> 
> diff --git a/tools/buildman/README b/tools/buildman/README
> index a5d181c..1c919af 100644
> --- a/tools/buildman/README
> +++ b/tools/buildman/README
> @@ -3,6 +3,8 @@
>  # SPDX-License-Identifier:	GPL-2.0+
>  #
>  
> +(Please read 'How to change from MAKEALL' if you are used to that tool)
> +
>  What is this?
>  =============
>  
> @@ -663,6 +665,96 @@ Other options
>  Buildman has various other command line options. Try --help to see them.
>  
>  
> +How to change from MAKEALL
> +==========================
> +
> +Buildman includes most of the features of MAKEALL and is generally faster
> +and easier to use. In particular it builds entire branches: if a particular
> +commit introduces an error in a particular board, buildman can easily show
> +you this, even if a later commit fixes that error.
> +
> +The reasons to deprecate MAKEALL are:
> +- We don't want to maintain two build systems
> +- Buildman is typically faster
> +- Buildman has a lot more features
> +
> +But still, many people will be sad to lose MAKEALL. If you are used to
> +MAKEALL, here are a few pointers.
> +First you need to set up your tool chains - see the 'Setting up' section
> +for details. Once you have your required toolchain(s) detected then you are
> +ready to go.
> +
> +Buildman works on entire branches, so the normal use is:
> +
> +   ./tools/buildman/buildman -b <branch_name> <list of things to build>
> +
> +followed by (afterwards, or perhaps concurrently in another terminal):
> +
> +   ./tools/buildman/buildman -b <branch_name> -s <list of things to build>


The biggest difference I see between buildman and MAKEALL is that
buildman tests commits in git-log,
whereas MAKEALL tests the current source tree.

It means buildman does not work for tarball users.
Maybe we can excuse because I guess most develpers are working on a git-repo.

But I find MAKEALL is much eaiser for a quick test.
It is very useful for my work flow;
Just change some code and invoke "./MAKEALL -s uniphier"
I can test local changes without commiting them.


In buildman, I need to commit the local changes once and do
git branch --set-upstream-to <upstream-branch>
tools/buildman/buildman -b <topic-branch>
which requires me lots of typing.


Best Regards
Masahiro Yamada

[U-Boot] [PATCH v2 2/3] buildman: Add some notes about moving from MAKEALL

[Simon Glass]

Hi Masahiro,

On 1 August 2014 02:10, Masahiro Yamada <yamada.m@jp.panasonic.com> wrote:
> Hi.
>
>
> On Mon, 28 Jul 2014 23:53:29 -0600
> Simon Glass <sjg@chromium.org> wrote:
>
>> For those used to MAKEALL, buildman seems strange. Add some notes to ease
>> the transition.
>>
>> Signed-off-by: Simon Glass <sjg@chromium.org>
>> ---
>>
>> Changes in v2:
>> - Minor changes to the text
>>
>>  tools/buildman/README | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 92 insertions(+)
>>
>> diff --git a/tools/buildman/README b/tools/buildman/README
>> index a5d181c..1c919af 100644
>> --- a/tools/buildman/README
>> +++ b/tools/buildman/README
>> @@ -3,6 +3,8 @@
>>  # SPDX-License-Identifier:   GPL-2.0+
>>  #
>>
>> +(Please read 'How to change from MAKEALL' if you are used to that tool)
>> +
>>  What is this?
>>  =============
>>
>> @@ -663,6 +665,96 @@ Other options
>>  Buildman has various other command line options. Try --help to see them.
>>
>>
>> +How to change from MAKEALL
>> +==========================
>> +
>> +Buildman includes most of the features of MAKEALL and is generally faster
>> +and easier to use. In particular it builds entire branches: if a particular
>> +commit introduces an error in a particular board, buildman can easily show
>> +you this, even if a later commit fixes that error.
>> +
>> +The reasons to deprecate MAKEALL are:
>> +- We don't want to maintain two build systems
>> +- Buildman is typically faster
>> +- Buildman has a lot more features
>> +
>> +But still, many people will be sad to lose MAKEALL. If you are used to
>> +MAKEALL, here are a few pointers.
>> +First you need to set up your tool chains - see the 'Setting up' section
>> +for details. Once you have your required toolchain(s) detected then you are
>> +ready to go.
>> +
>> +Buildman works on entire branches, so the normal use is:
>> +
>> +   ./tools/buildman/buildman -b <branch_name> <list of things to build>
>> +
>> +followed by (afterwards, or perhaps concurrently in another terminal):
>> +
>> +   ./tools/buildman/buildman -b <branch_name> -s <list of things to build>
>
>
> The biggest difference I see between buildman and MAKEALL is that
> buildman tests commits in git-log,
> whereas MAKEALL tests the current source tree.
>
> It means buildman does not work for tarball users.
> Maybe we can excuse because I guess most develpers are working on a git-repo.
>
> But I find MAKEALL is much eaiser for a quick test.
> It is very useful for my work flow;
> Just change some code and invoke "./MAKEALL -s uniphier"
> I can test local changes without commiting them.
>
>
> In buildman, I need to commit the local changes once and do
> git branch --set-upstream-to <upstream-branch>
> tools/buildman/buildman -b <topic-branch>
> which requires me lots of typing.

OK thanks for pointing that out. That feature existed in an early
pre-submission version of buildman but I dropped it. I'll see if I can
implement ti again.

Regards,
Simon

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Tom Rini]

On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:

> Buildman has been around for a little over a year and is used by a fair
> number of U-Boot developers. However quite a few people still use MAKEALL.
> 
> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
> start that process.

OK, lets start.  First, I had to make up a special directory with
symlinks to the various toolchains, in order to get them to work with
buildman, due to the various layouts:

$ ls -l ~/work/u-boot/toolchain-links/
total 4
lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4

I tried a few things but I couldn't get a toolchain section setup that
found all of those.  Second:
(testing the MMC pull request)
$ ./tools/buildman/buildman -b master
'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
No section: 'make-flags'
Building 11 commits for 121 boards (6 threads, 1 job per thread)
Cloning repo for thread 0
Cloning repo for thread 1
Cloning repo for thread 2
Cloning repo for thread 3
Cloning repo for thread 4
Cloning repo for thread 5
 1155   88   88 /1331   axs101

So it's nice that it spit out a thread per commit, but I really only
care right now if master builds right.  And I've gone blind and can't
find the end result logs of what failed to build (88 failures doesn't
quite match the warning/errors I get for my wrapper around
toolchains+MAKEALL).

I also didn't quite see how to plugin an arm64 toolchain.

Next up, is there some way to have multiple toolchains for an arch
available?  On some compute resources I have at work I throw
arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
for some projects) and some various combos for powerpc as well.  I
didn't see how to do that with buildman aside from different ~/.buildman
files per invocation.

Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140804/a101f6fa/attachment.pgp>

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Simon Glass]

Hi Tom,

On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
> On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
>
>> Buildman has been around for a little over a year and is used by a fair
>> number of U-Boot developers. However quite a few people still use MAKEALL.
>>
>> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
>> start that process.
>
> OK, lets start.  First, I had to make up a special directory with
> symlinks to the various toolchains, in order to get them to work with
> buildman, due to the various layouts:
>
> $ ls -l ~/work/u-boot/toolchain-links/
> total 4
> lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
> lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
> lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
> lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
> lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
> lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
> lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
> lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
> lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4
>
> I tried a few things but I couldn't get a toolchain section setup that
> found all of those.  Second:

OK, can you give me a trace of the output of 'buildman
--list-tool-chains' - I should be able to figure out what is wrong.

> (testing the MMC pull request)
> $ ./tools/buildman/buildman -b master
> 'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
> No section: 'make-flags'
> Building 11 commits for 121 boards (6 threads, 1 job per thread)
> Cloning repo for thread 0
> Cloning repo for thread 1
> Cloning repo for thread 2
> Cloning repo for thread 3
> Cloning repo for thread 4
> Cloning repo for thread 5
>  1155   88   88 /1331   axs101
>
> So it's nice that it spit out a thread per commit, but I really only
> care right now if master builds right.  And I've gone blind and can't
> find the end result logs of what failed to build (88 failures doesn't
> quite match the warning/errors I get for my wrapper around
> toolchains+MAKEALL).

You can use the -s flag to see the broken builds, and -se to see
errors also. Is that what you mean?

The output is in ../<branch>/<commit>/<board> but if you want to keep
the binaries, you need -k also.

>
> I also didn't quite see how to plugin an arm64 toolchain.

I haven't tried - can you please point me to one and I'll give it a go.

>
> Next up, is there some way to have multiple toolchains for an arch
> available?  On some compute resources I have at work I throw
> arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
> for some projects) and some various combos for powerpc as well.  I
> didn't see how to do that with buildman aside from different ~/.buildman
> files per invocation.

Yes I remember you talking about this before. One option would be to
add toolchain as a new dimension in the matrix. It might be painful
though. So to be clear, you want to define multiple toolchains for an
arch, and have buildman always build with all of them?

Regards,
Simon

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Jeroen Hofstee]

Hello Simon,

On 04-08-14 22:41, Simon Glass wrote:
> Hi Tom,
>
> On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
>> On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
>>
>>> Buildman has been around for a little over a year and is used by a fair
>>> number of U-Boot developers. However quite a few people still use MAKEALL.
>>>
>>> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
>>> start that process.
>> OK, lets start.  First, I had to make up a special directory with
>> symlinks to the various toolchains, in order to get them to work with
>> buildman, due to the various layouts:
>>
>> $ ls -l ~/work/u-boot/toolchain-links/
>> total 4
>> lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
>> lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
>> lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
>> lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
>> lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
>> lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
>> lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
>> lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
>> lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4
>>
>> I tried a few things but I couldn't get a toolchain section setup that
>> found all of those.  Second:
> OK, can you give me a trace of the output of 'buildman
> --list-tool-chains' - I should be able to figure out what is wrong.

Like Tom, I tried this a bit. (and ended up faking clang is gcc). There
are at least some assumptions cc = gcc and strlen(cc) == 3.
>> (testing the MMC pull request)
>> $ ./tools/buildman/buildman -b master
>> 'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
>> No section: 'make-flags'
>> Building 11 commits for 121 boards (6 threads, 1 job per thread)
>> Cloning repo for thread 0
>> Cloning repo for thread 1
>> Cloning repo for thread 2
>> Cloning repo for thread 3
>> Cloning repo for thread 4
>> Cloning repo for thread 5
>>   1155   88   88 /1331   axs101
>>
>> So it's nice that it spit out a thread per commit, but I really only
>> care right now if master builds right.  And I've gone blind and can't
>> find the end result logs of what failed to build (88 failures doesn't
>> quite match the warning/errors I get for my wrapper around
>> toolchains+MAKEALL).
> You can use the -s flag to see the broken builds, and -se to see
> errors also. Is that what you mean?
>
> The output is in ../<branch>/<commit>/<board> but if you want to keep
> the binaries, you need -k also.

Like Tom, I saw fancy output, like merged not being a property
or something. I like to see the warnings and errors directly, but -e?
failed to do so. (or at least I didn't spend enough time to get it to work)
>> I also didn't quite see how to plugin an arm64 toolchain.
> I haven't tried - can you please point me to one and I'll give it a go.
>
>> Next up, is there some way to have multiple toolchains for an arch
>> available?  On some compute resources I have at work I throw
>> arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
>> for some projects) and some various combos for powerpc as well.  I
>> didn't see how to do that with buildman aside from different ~/.buildman
>> files per invocation.
> Yes I remember you talking about this before. One option would be to
> add toolchain as a new dimension in the matrix. It might be painful
> though. So to be clear, you want to define multiple toolchains for an
> arch, and have buildman always build with all of them?
>

I would like this as well. For serious changing I typically build with
MAKEALL with clang and gcc (for ARM) + gcc for powerpc if it is
a overall change.

But since clang always warns it is actually not a viable target for
buildman at all.

Regards,
Jeroen

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Tom Rini]

On Mon, Aug 04, 2014 at 02:41:45PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
> > On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
> >
> >> Buildman has been around for a little over a year and is used by a fair
> >> number of U-Boot developers. However quite a few people still use MAKEALL.
> >>
> >> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
> >> start that process.
> >
> > OK, lets start.  First, I had to make up a special directory with
> > symlinks to the various toolchains, in order to get them to work with
> > buildman, due to the various layouts:
> >
> > $ ls -l ~/work/u-boot/toolchain-links/
> > total 4
> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
> > lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
> > lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
> > lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
> > lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
> > lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4
> >
> > I tried a few things but I couldn't get a toolchain section setup that
> > found all of those.  Second:
> 
> OK, can you give me a trace of the output of 'buildman
> --list-tool-chains' - I should be able to figure out what is wrong.

Sure but I think the problem was I couldn't get it to recurse all the
way down, just a few levels (which isn't a bad thing exactly, it'd be
going pretty deep otherwise).

> > (testing the MMC pull request)
> > $ ./tools/buildman/buildman -b master
> > 'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
> > No section: 'make-flags'
> > Building 11 commits for 121 boards (6 threads, 1 job per thread)
> > Cloning repo for thread 0
> > Cloning repo for thread 1
> > Cloning repo for thread 2
> > Cloning repo for thread 3
> > Cloning repo for thread 4
> > Cloning repo for thread 5
> >  1155   88   88 /1331   axs101
> >
> > So it's nice that it spit out a thread per commit, but I really only
> > care right now if master builds right.  And I've gone blind and can't
> > find the end result logs of what failed to build (88 failures doesn't
> > quite match the warning/errors I get for my wrapper around
> > toolchains+MAKEALL).
> 
> You can use the -s flag to see the broken builds, and -se to see
> errors also. Is that what you mean?

That's closer, yeah.  Oh, and -c 1 seems to do be what I want, except I
get:
$ ./tools/buildman/buildman -se 'blackfin' -c 1
No section: 'make-flags'
Traceback (most recent call last):
  File "./tools/buildman/buildman", line 135, in <module>
    control.DoBuildman(options, args)
  File "/home/trini/work/u-boot/u-boot/tools/buildman/control.py", line 138, in DoBuildman
    1)
  File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/patchstream.py", line 389, in GetMetaDataForList
    stdout = command.RunPipe(pipe, capture=True).stdout
  File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/command.py", line 50, in RunPipe
    user_pipestr =  '|'.join([' '.join(pipe) for pipe in pipe_list])
TypeError: sequence item 8: expected string, NoneType found

> The output is in ../<branch>/<commit>/<board> but if you want to keep
> the binaries, you need -k also.

Ah, OK.

> > I also didn't quite see how to plugin an arm64 toolchain.
> 
> I haven't tried - can you please point me to one and I'll give it a go.

There should be something newer than
https://launchpad.net/linaro-toolchain-binaries but I'm failing to find
it quickly.

> > Next up, is there some way to have multiple toolchains for an arch
> > available?  On some compute resources I have at work I throw
> > arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
> > for some projects) and some various combos for powerpc as well.  I
> > didn't see how to do that with buildman aside from different ~/.buildman
> > files per invocation.
> 
> Yes I remember you talking about this before. One option would be to
> add toolchain as a new dimension in the matrix. It might be painful
> though. So to be clear, you want to define multiple toolchains for an
> arch, and have buildman always build with all of them?

Well, a --config-file= would be good enough so I could have
buildman-eldk54.cfg, buildman-eldk42.cfg, etc, etc.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140804/d612919b/attachment.pgp>

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Simon Glass]

Hi Tom,

On 4 August 2014 15:23, Tom Rini <trini@ti.com> wrote:
> On Mon, Aug 04, 2014 at 02:41:45PM -0600, Simon Glass wrote:
>> Hi Tom,
>>
>> On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
>> > On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
>> >
>> >> Buildman has been around for a little over a year and is used by a fair
>> >> number of U-Boot developers. However quite a few people still use MAKEALL.
>> >>
>> >> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
>> >> start that process.
>> >
>> > OK, lets start.  First, I had to make up a special directory with
>> > symlinks to the various toolchains, in order to get them to work with
>> > buildman, due to the various layouts:
>> >
>> > $ ls -l ~/work/u-boot/toolchain-links/
>> > total 4
>> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
>> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
>> > lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
>> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
>> > lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
>> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
>> > lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
>> > lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
>> > lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4
>> >
>> > I tried a few things but I couldn't get a toolchain section setup that
>> > found all of those.  Second:
>>
>> OK, can you give me a trace of the output of 'buildman
>> --list-tool-chains' - I should be able to figure out what is wrong.
>
> Sure but I think the problem was I couldn't get it to recurse all the
> way down, just a few levels (which isn't a bad thing exactly, it'd be
> going pretty deep otherwise).

It looks at the path you give, and then appends bin/ and usr/bin as
well. Then it looks for something ending in gcc in each directory. You
may need to specify additional search paths to make this work. My
output with ~/.buildman:

# Buildman settings file

[toolchain]
root: /
#rest: /home/sjg/c/buildall/toolchains/*
#eldk: /opt/eldk4.2/usr/bin
#arm: /home/sjg/c/tools/arm-unknown-eabi
avr32: /home/sjg/c/buildall/toolchains/avr32
#m68k: /home/sjg/c/buildall/toolchains/m68k-linux
m68k: /usr/local/m68k-uclinux-tools
arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
eldk: /opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux

[toolchain-alias]
x86: i386

[make-flags]
src=/home/sjg/c/src
chroot=/home/sjg/c/chroot
vboot=USE_STDINT=1 VBOOT_DEBUG=1 MAKEFLAGS_VBOOT=DEBUG=1 CFLAGS_EXTRA_VBOOT=-DUN
ROLL_LOOPS VBOOT_SOURCE=${src}/platform/vboot_reference
chromeos_coreboot=VBOOT=${chroot}/build/link/usr ${vboot}
chromeos_daisy=VBOOT=${chroot}/build/daisy/usr ${vboot}

is:

./tools/buildman/buildman --list-tool-chains
Scanning for tool chains
   - scanning path '/'
      - looking in '/.'
      - looking in '/bin'
      - looking in '/usr/bin'
         - found '/usr/bin/gcc'
Tool chain test:  OK
         - found '/usr/bin/c89-gcc'
Tool chain test:  OK
         - found '/usr/bin/c99-gcc'
Tool chain test:  OK
         - found '/usr/bin/x86_64-linux-gnu-gcc'
Tool chain test:  OK
   - scanning path '/home/sjg/c/buildall/toolchains/avr32'
      - looking in '/home/sjg/c/buildall/toolchains/avr32/.'
      - looking in '/home/sjg/c/buildall/toolchains/avr32/bin'
      - looking in '/home/sjg/c/buildall/toolchains/avr32/usr/bin'
         - found
'/home/sjg/c/buildall/toolchains/avr32/usr/bin/avr32-buildroot-linux-uclibc-gcc'
Tool chain test:  OK
         - found '/home/sjg/c/buildall/toolchains/avr32/usr/bin/avr32-linux-gcc'
Tool chain test:  OK
   - scanning path '/usr/local/m68k-uclinux-tools'
      - looking in '/usr/local/m68k-uclinux-tools/.'
      - looking in '/usr/local/m68k-uclinux-tools/bin'
      - looking in '/usr/local/m68k-uclinux-tools/usr/bin'
   - scanning path
'/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux'
      - looking in
'/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/.'
      - looking in
'/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin'
         - found
'/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm-linux-gnueabihf-gcc'
Tool chain test:  OK
      - looking in
'/opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/usr/bin'
   - scanning path
'/opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux'
      - looking in
'/opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux/.'
         - found
'/opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux/./powerpc-linux-gcc'
Tool chain test:  OK
         - found
'/opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux/./powerpc-gcc'
Tool chain test:  OK
      - looking in
'/opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux/bin'
      - looking in
'/opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux/usr/bin'
List of available toolchains (7):
arm       : /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux/bin/arm-linux-gnueabihf-gcc
avr32     : /home/sjg/c/buildall/toolchains/avr32/usr/bin/avr32-buildroot-linux-uclibc-gcc
c89       : /usr/bin/c89-gcc
c99       : /usr/bin/c99-gcc
powerpc   : /opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux/./powerpc-linux-gcc
sandbox   : /usr/bin/gcc
x86_64    : /usr/bin/x86_64-linux-gnu-gcc

>
>> > (testing the MMC pull request)
>> > $ ./tools/buildman/buildman -b master
>> > 'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
>> > No section: 'make-flags'
>> > Building 11 commits for 121 boards (6 threads, 1 job per thread)
>> > Cloning repo for thread 0
>> > Cloning repo for thread 1
>> > Cloning repo for thread 2
>> > Cloning repo for thread 3
>> > Cloning repo for thread 4
>> > Cloning repo for thread 5
>> >  1155   88   88 /1331   axs101
>> >
>> > So it's nice that it spit out a thread per commit, but I really only
>> > care right now if master builds right.  And I've gone blind and can't
>> > find the end result logs of what failed to build (88 failures doesn't
>> > quite match the warning/errors I get for my wrapper around
>> > toolchains+MAKEALL).
>>
>> You can use the -s flag to see the broken builds, and -se to see
>> errors also. Is that what you mean?
>
> That's closer, yeah.  Oh, and -c 1 seems to do be what I want, except I
> get:
> $ ./tools/buildman/buildman -se 'blackfin' -c 1
> No section: 'make-flags'
> Traceback (most recent call last):
>   File "./tools/buildman/buildman", line 135, in <module>
>     control.DoBuildman(options, args)
>   File "/home/trini/work/u-boot/u-boot/tools/buildman/control.py", line 138, in DoBuildman
>     1)
>   File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/patchstream.py", line 389, in GetMetaDataForList
>     stdout = command.RunPipe(pipe, capture=True).stdout
>   File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/command.py", line 50, in RunPipe
>     user_pipestr =  '|'.join([' '.join(pipe) for pipe in pipe_list])
> TypeError: sequence item 8: expected string, NoneType found

That's because it doesn't have a branch (-b <branch>). If you are just
wanting to build the current checked-out tree, you could apply the v3
series which supports that.

>
>> The output is in ../<branch>/<commit>/<board> but if you want to keep
>> the binaries, you need -k also.
>
> Ah, OK.
>
>> > I also didn't quite see how to plugin an arm64 toolchain.
>>
>> I haven't tried - can you please point me to one and I'll give it a go.
>
> There should be something newer than
> https://launchpad.net/linaro-toolchain-binaries but I'm failing to find
> it quickly.

OK I'll take a look.

>
>> > Next up, is there some way to have multiple toolchains for an arch
>> > available?  On some compute resources I have at work I throw
>> > arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
>> > for some projects) and some various combos for powerpc as well.  I
>> > didn't see how to do that with buildman aside from different ~/.buildman
>> > files per invocation.
>>
>> Yes I remember you talking about this before. One option would be to
>> add toolchain as a new dimension in the matrix. It might be painful
>> though. So to be clear, you want to define multiple toolchains for an
>> arch, and have buildman always build with all of them?
>
> Well, a --config-file= would be good enough so I could have
> buildman-eldk54.cfg, buildman-eldk42.cfg, etc, etc.

OK I'll add a new patch to a v4 series to implement that. It's pretty
trivial so should get to it today.

Regards,
Simon

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Simon Glass]

Hi Tom,

On 5 August 2014 08:06, Simon Glass <sjg@chromium.org> wrote:
> Hi Tom,
>
> On 4 August 2014 15:23, Tom Rini <trini@ti.com> wrote:
>> On Mon, Aug 04, 2014 at 02:41:45PM -0600, Simon Glass wrote:
>>> Hi Tom,
>>>
>>> On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
[ship]

>>> > I also didn't quite see how to plugin an arm64 toolchain.
>>>
>>> I haven't tried - can you please point me to one and I'll give it a go.
>>
>> There should be something newer than
>> https://launchpad.net/linaro-toolchain-binaries but I'm failing to find
>> it quickly.
>
> OK I'll take a look.

This line in ~/.buildman works for me:

aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux

Regards,
Simon

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Tom Rini]

On Tue, Aug 05, 2014 at 08:06:25AM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On 4 August 2014 15:23, Tom Rini <trini@ti.com> wrote:
> > On Mon, Aug 04, 2014 at 02:41:45PM -0600, Simon Glass wrote:
> >> Hi Tom,
> >>
> >> On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
> >> > On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
> >> >
> >> >> Buildman has been around for a little over a year and is used by a fair
> >> >> number of U-Boot developers. However quite a few people still use MAKEALL.
> >> >>
> >> >> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
> >> >> start that process.
> >> >
> >> > OK, lets start.  First, I had to make up a special directory with
> >> > symlinks to the various toolchains, in order to get them to work with
> >> > buildman, due to the various layouts:
> >> >
> >> > $ ls -l ~/work/u-boot/toolchain-links/
> >> > total 4
> >> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
> >> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
> >> > lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
> >> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
> >> > lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
> >> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
> >> > lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
> >> > lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
> >> > lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4
> >> >
> >> > I tried a few things but I couldn't get a toolchain section setup that
> >> > found all of those.  Second:
> >>
> >> OK, can you give me a trace of the output of 'buildman
> >> --list-tool-chains' - I should be able to figure out what is wrong.
> >
> > Sure but I think the problem was I couldn't get it to recurse all the
> > way down, just a few levels (which isn't a bad thing exactly, it'd be
> > going pretty deep otherwise).
> 
> It looks at the path you give, and then appends bin/ and usr/bin as
> well. Then it looks for something ending in gcc in each directory. You
> may need to specify additional search paths to make this work. My
> output with ~/.buildman:
> 
> # Buildman settings file
> 
> [toolchain]
> root: /
> #rest: /home/sjg/c/buildall/toolchains/*
> #eldk: /opt/eldk4.2/usr/bin
> #arm: /home/sjg/c/tools/arm-unknown-eabi
> avr32: /home/sjg/c/buildall/toolchains/avr32
> #m68k: /home/sjg/c/buildall/toolchains/m68k-linux
> m68k: /usr/local/m68k-uclinux-tools
> arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
> eldk: /opt/eldk-5.2.1/powerpc/sysroots/i686-eldk-linux/usr/bin/powerpc-linux

Ah-ha, OK.  I think we need to update the README slightly, I was
thinking it was just a few keywords based on how it is now.  So I would
need to re-work my config similar to yours and then yes, it would find
everything as well.

[snip]
> > That's closer, yeah.  Oh, and -c 1 seems to do be what I want, except I
> > get:
> > $ ./tools/buildman/buildman -se 'blackfin' -c 1
> > No section: 'make-flags'
> > Traceback (most recent call last):
> >   File "./tools/buildman/buildman", line 135, in <module>
> >     control.DoBuildman(options, args)
> >   File "/home/trini/work/u-boot/u-boot/tools/buildman/control.py", line 138, in DoBuildman
> >     1)
> >   File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/patchstream.py", line 389, in GetMetaDataForList
> >     stdout = command.RunPipe(pipe, capture=True).stdout
> >   File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/command.py", line 50, in RunPipe
> >     user_pipestr =  '|'.join([' '.join(pipe) for pipe in pipe_list])
> > TypeError: sequence item 8: expected string, NoneType found
> 
> That's because it doesn't have a branch (-b <branch>). If you are just
> wanting to build the current checked-out tree, you could apply the v3
> series which supports that.

ah, ok.

> >> > Next up, is there some way to have multiple toolchains for an arch
> >> > available?  On some compute resources I have at work I throw
> >> > arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
> >> > for some projects) and some various combos for powerpc as well.  I
> >> > didn't see how to do that with buildman aside from different ~/.buildman
> >> > files per invocation.
> >>
> >> Yes I remember you talking about this before. One option would be to
> >> add toolchain as a new dimension in the matrix. It might be painful
> >> though. So to be clear, you want to define multiple toolchains for an
> >> arch, and have buildman always build with all of them?
> >
> > Well, a --config-file= would be good enough so I could have
> > buildman-eldk54.cfg, buildman-eldk42.cfg, etc, etc.
> 
> OK I'll add a new patch to a v4 series to implement that. It's pretty
> trivial so should get to it today.

Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140805/ab2903a0/attachment.pgp>

[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Simon Glass]

Hi Jeroen,

On 4 August 2014 14:56, Jeroen Hofstee <jeroen@myspectrum.nl> wrote:
> Hello Simon,
>
>
> On 04-08-14 22:41, Simon Glass wrote:
>>
>> Hi Tom,
>>
>> On 4 August 2014 07:23, Tom Rini <trini@ti.com> wrote:
>>>
>>> On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
>>>
>>>> Buildman has been around for a little over a year and is used by a fair
>>>> number of U-Boot developers. However quite a few people still use
>>>> MAKEALL.
>>>>
>>>> Buildman was intended to replace MAKEALL, so perhaps now is a good time
>>>> to
>>>> start that process.
>>>
>>> OK, lets start.  First, I had to make up a special directory with
>>> symlinks to the various toolchains, in order to get them to work with
>>> buildman, due to the various layouts:
>>>
>>> $ ls -l ~/work/u-boot/toolchain-links/
>>> total 4
>>> lrwxrwxrwx 1 trini trini 38 Aug  1 17:09
>>> arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
>>> lrwxrwxrwx 1 trini trini 38 Aug  1 17:09
>>> arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
>>> lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf ->
>>> /opt/uClinux-45/bfin-elf
>>> lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux ->
>>> /opt/gcc-4.9.0-nolibc/m68k-linux
>>> lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu ->
>>> /opt/microblazeel-xilinx-linux-gnu
>>> lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu ->
>>> /opt/microblaze-xilinx-linux-gnu
>>> lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux ->
>>> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
>>> lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f ->
>>> /opt/nds32le/nds32le-linux-glibc-v1f
>>> lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 ->
>>> /opt/sparc-elf-3.4.4
>>>
>>> I tried a few things but I couldn't get a toolchain section setup that
>>> found all of those.  Second:
>>
>> OK, can you give me a trace of the output of 'buildman
>> --list-tool-chains' - I should be able to figure out what is wrong.
>
>
> Like Tom, I tried this a bit. (and ended up faking clang is gcc). There
> are at least some assumptions cc = gcc and strlen(cc) == 3.

OK I will change that in v4.

>
>>> (testing the MMC pull request)
>>> $ ./tools/buildman/buildman -b master
>>> 'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
>>> No section: 'make-flags'
>>> Building 11 commits for 121 boards (6 threads, 1 job per thread)
>>> Cloning repo for thread 0
>>> Cloning repo for thread 1
>>> Cloning repo for thread 2
>>> Cloning repo for thread 3
>>> Cloning repo for thread 4
>>> Cloning repo for thread 5
>>>   1155   88   88 /1331   axs101
>>>
>>> So it's nice that it spit out a thread per commit, but I really only
>>> care right now if master builds right.  And I've gone blind and can't
>>> find the end result logs of what failed to build (88 failures doesn't
>>> quite match the warning/errors I get for my wrapper around
>>> toolchains+MAKEALL).
>>
>> You can use the -s flag to see the broken builds, and -se to see
>> errors also. Is that what you mean?
>>
>> The output is in ../<branch>/<commit>/<board> but if you want to keep
>> the binaries, you need -k also.
>
>
> Like Tom, I saw fancy output, like merged not being a property
> or something. I like to see the warnings and errors directly, but -e?
> failed to do so. (or at least I didn't spend enough time to get it to work)

With the v3 series you can use the -v option to display all output
while it builds. The -e option displays summarised information,
interned to avoid you getting the same error 100 times for 100
different boards.

>
>>> I also didn't quite see how to plugin an arm64 toolchain.
>>
>> I haven't tried - can you please point me to one and I'll give it a go.
>>
>>> Next up, is there some way to have multiple toolchains for an arch
>>> available?  On some compute resources I have at work I throw
>>> arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
>>> for some projects) and some various combos for powerpc as well.  I
>>> didn't see how to do that with buildman aside from different ~/.buildman
>>> files per invocation.
>>
>> Yes I remember you talking about this before. One option would be to
>> add toolchain as a new dimension in the matrix. It might be painful
>> though. So to be clear, you want to define multiple toolchains for an
>> arch, and have buildman always build with all of them?
>>
>
> I would like this as well. For serious changing I typically build with
> MAKEALL with clang and gcc (for ARM) + gcc for powerpc if it is
> a overall change.

OK I'll add a config file option.

>
> But since clang always warns it is actually not a viable target for
> buildman at all.

Why does it always warn?

Regards,
Simon