* [U-Boot] [PATCH v6 01/20] buildman: Fix a few typos
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
@ 2014-08-09 21:32 ` Simon Glass
2014-08-09 21:32 ` [U-Boot] [PATCH v6 02/20] buildman: Add some notes about moving from MAKEALL Simon Glass
` (18 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:32 UTC (permalink / raw)
To: u-boot
There are several typos in the README - fix them.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5:
- Drop patch to search for *cc instead of *gcc for the compiler
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/README | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/tools/buildman/README b/tools/buildman/README
index c30c1d4..a5d181c 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -71,8 +71,8 @@ directory. It clones this repository into a copy for each thread, and the
threads do not affect the state of your git repository. Any checkouts done
by the thread affect only the working directory for that thread.
-Buildman automatically selects the correct toolchain for each board. You
-must supply suitable toolchains, but buildman takes care of selecting the
+Buildman automatically selects the correct tool chain for each board. You
+must supply suitable tool chains, but buildman takes care of selecting the
right one.
Buildman always builds a branch, and always builds the upstream commit as
@@ -287,7 +287,7 @@ If it can't detect the upstream branch, try checking out the branch, and
doing something like 'git branch --set-upstream <branch> upstream/master'
or something similar.
-As an exmmple:
+As an example:
Dry run, so not doing much. But I would do this:
@@ -339,7 +339,7 @@ Building 18 commits for 1059 boards (4 threads, 1 job per thread)
528 36 124 /19062 1:13:30 : SIMPC8313_SP
This means that it is building 19062 board/commit combinations. So far it
-has managed to succesfully build 528. Another 36 have built with warnings,
+has managed to successfully build 528. Another 36 have built with warnings,
and 124 more didn't build at all. Buildman expects to complete the process
in an hour and 15 minutes. Use this time to buy a faster computer.
@@ -413,7 +413,7 @@ again.
At commit 16, the error moves - you can see that the old error at line 120
is fixed, but there is a new one at line 126. This is probably only because
-we added some code and moved the broken line futher down the file.
+we added some code and moved the broken line father down the file.
If many boards have the same error, then -e will display the error only
once. This makes the output as concise as possible.
@@ -491,7 +491,7 @@ You can also use -d to see a detailed size breakdown for each board. This
list is sorted in order from largest growth to largest reduction.
It is possible to go a little further with the -B option (--bloat). This
-shows where U-Boot has bloted, breaking the size change down to the function
+shows where U-Boot has bloated, breaking the size change down to the function
level. Example output is below:
$ ./tools/buildman/buildman -b us-mem4 -sSdB
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 02/20] buildman: Add some notes about moving from MAKEALL
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
2014-08-09 21:32 ` [U-Boot] [PATCH v6 01/20] buildman: Fix a few typos Simon Glass
@ 2014-08-09 21:32 ` Simon Glass
2014-08-09 21:32 ` [U-Boot] [PATCH v6 03/20] buildman: Allow building of current source tree Simon Glass
` (17 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:32 UTC (permalink / raw)
To: u-boot
For those used to MAKEALL, buildman seems strange. Add some notes to ease
the transition.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to remove patman's -a option
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/README | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 92 insertions(+)
diff --git a/tools/buildman/README b/tools/buildman/README
index a5d181c..1c919af 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -3,6 +3,8 @@
# SPDX-License-Identifier: GPL-2.0+
#
+(Please read 'How to change from MAKEALL' if you are used to that tool)
+
What is this?
=============
@@ -663,6 +665,96 @@ Other options
Buildman has various other command line options. Try --help to see them.
+How to change from MAKEALL
+==========================
+
+Buildman includes most of the features of MAKEALL and is generally faster
+and easier to use. In particular it builds entire branches: if a particular
+commit introduces an error in a particular board, buildman can easily show
+you this, even if a later commit fixes that error.
+
+The reasons to deprecate MAKEALL are:
+- We don't want to maintain two build systems
+- Buildman is typically faster
+- Buildman has a lot more features
+
+But still, many people will be sad to lose MAKEALL. If you are used to
+MAKEALL, here are a few pointers.
+
+First you need to set up your tool chains - see the 'Setting up' section
+for details. Once you have your required toolchain(s) detected then you are
+ready to go.
+
+Buildman works on entire branches, so the normal use is:
+
+ ./tools/buildman/buildman -b <branch_name> <list of things to build>
+
+followed by (afterwards, or perhaps concurrently in another terminal):
+
+ ./tools/buildman/buildman -b <branch_name> -s <list of things to build>
+
+to see the results of the build. Rather than showing you all the output,
+buildman just shows a summary, with red indicating that a commit introduced
+an error and green indicating that a commit fixed an error. Use the -e
+flag to see the full errors.
+
+You don't need to stick around on that branch while buildman is running. It
+checks out its own copy of the source code, so you can change branches,
+add commits, etc. without affecting the build in progress.
+
+The <list of things to build> can include board names, architectures or the
+like. There are no flags to disambiguate since ambiguities are rare. Using
+the examples from MAKEALL:
+
+Examples:
+ - build all Power Architecture boards:
+ MAKEALL -a powerpc
+ MAKEALL --arch powerpc
+ MAKEALL powerpc
+ ** buildman -b <branch> powerpc
+ - build all PowerPC boards manufactured by vendor "esd":
+ MAKEALL -a powerpc -v esd
+ ** buildman -b <branch> esd
+ - build all PowerPC boards manufactured either by "keymile" or "siemens":
+ MAKEALL -a powerpc -v keymile -v siemens
+ ** buildman -b <branch> keymile siemens
+ - build all Freescale boards with MPC83xx CPUs, plus all 4xx boards:
+ MAKEALL -c mpc83xx -v freescale 4xx
+ ** buildman -b <branch> mpc83xx freescale 4xx
+
+Buildman automatically tries to use all the CPUs in your machine. If you
+are building a lot of boards it will use one thread for every CPU core
+it detects in your machine. This is like MAKEALL's BUILD_NBUILDS option.
+You can use the -T flag to change the number of threads. If you are only
+building a few boards, buildman will automatically run make with the -j
+flag to increase the number of concurrent make tasks. It isn't normally
+that helpful to fiddle with this option, but if you use the BUILD_NCPUS
+option in MAKEALL then -j is the equivalent in buildman.
+
+Buildman puts its output in ../<branch_name> by default but you can change
+this with the -o option. Buildman normally does out-of-tree builds: use -i
+to disable that if you really want to. But be careful that once you have
+used -i you pollute buildman's copies of the source tree, and you will need
+to remove the build directory (normally ../<branch_name>) to run buildman
+in normal mode (without -i).
+
+Buildman doesn't keep the output result normally, but use the -k option to
+do this.
+
+Please read 'Theory of Operation' a few times as it will make a lot of
+things clearer.
+
+Some options you might like are:
+
+ -B shows which functions are growing/shrinking in which commit - great
+ for finding code bloat.
+ -S shows image sizes for each commit (just an overall summary)
+ -u shows boards that you haven't built yet
+ --step 0 will build just the upstream commit and the last commit of your
+ branch. This is often a quick sanity check that your branch doesn't
+ break anything. But note this does not check bisectability!
+
+
TODO
====
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 03/20] buildman: Allow building of current source tree
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
2014-08-09 21:32 ` [U-Boot] [PATCH v6 01/20] buildman: Fix a few typos Simon Glass
2014-08-09 21:32 ` [U-Boot] [PATCH v6 02/20] buildman: Add some notes about moving from MAKEALL Simon Glass
@ 2014-08-09 21:32 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 04/20] buildman: Move BuilderThread code to its own file Simon Glass
` (16 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:32 UTC (permalink / raw)
To: u-boot
Originally buildman had some support for building the current source tree.
However this was dropped before it was submitted, as part of the effort to
make it faster when building entire branches.
Reinstate this support. If no -b option is given, buildman will build the
current source tree.
Reported-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to fix patman unit tests
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/builder.py | 71 +++++++++++++++++++++++++-----------
tools/buildman/control.py | 91 ++++++++++++++++++++++++++++++-----------------
2 files changed, 108 insertions(+), 54 deletions(-)
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index 48408ff..d2b72d5 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -248,21 +248,34 @@ class BuilderThread(threading.Thread):
if self.toolchain:
# Checkout the right commit
- if commit_upto is not None:
+ if self.builder.commits:
commit = self.builder.commits[commit_upto]
if self.builder.checkout:
git_dir = os.path.join(work_dir, '.git')
gitutil.Checkout(commit.hash, git_dir, work_dir,
force=True)
else:
- commit = self.builder.commit # Ick, fix this for BuildCommits()
+ commit = 'current'
# Set up the environment and command line
env = self.toolchain.MakeEnvironment()
Mkdir(out_dir)
args = []
+ cwd = work_dir
if not self.builder.in_tree:
- args.append('O=build')
+ if commit_upto is None:
+ # In this case we are building in the original source
+ # directory (i.e. the current directory where buildman
+ # is invoked. The output directory is set to this
+ # thread's selected work directory.
+ #
+ # Symlinks can confuse U-Boot's Makefile since
+ # we may use '..' in our path, so remove them.
+ work_dir = os.path.realpath(work_dir)
+ args.append('O=%s/build' % work_dir)
+ cwd = None
+ else:
+ args.append('O=build')
args.append('-s')
if self.builder.num_jobs is not None:
args.extend(['-j', str(self.builder.num_jobs)])
@@ -272,14 +285,14 @@ class BuilderThread(threading.Thread):
# If we need to reconfigure, do that now
if do_config:
- result = self.Make(commit, brd, 'distclean', work_dir,
+ result = self.Make(commit, brd, 'distclean', cwd,
'distclean', *args, env=env)
- result = self.Make(commit, brd, 'config', work_dir,
+ result = self.Make(commit, brd, 'config', cwd,
*(args + config_args), env=env)
config_out = result.combined
do_config = False # No need to configure next time
if result.return_code == 0:
- result = self.Make(commit, brd, 'build', work_dir, *args,
+ result = self.Make(commit, brd, 'build', cwd, *args,
env=env)
result.stdout = config_out + result.stdout
else:
@@ -478,8 +491,10 @@ class BuilderThread(threading.Thread):
self.builder.out_queue.put(result)
else:
# Just build the currently checked-out build
- result = self.RunCommit(None, True)
- result.commit_upto = self.builder.upto
+ result, request_config = self.RunCommit(None, brd, work_dir, True,
+ True, self.builder.force_build_failures)
+ result.commit_upto = 0
+ self._WriteResult(result, job.keep_outputs)
self.builder.out_queue.put(result)
def run(self):
@@ -491,12 +506,16 @@ class BuilderThread(threading.Thread):
alive = True
while True:
job = self.builder.queue.get()
+ if self.builder.active and alive:
+ self.RunJob(job)
+ '''
try:
if self.builder.active and alive:
self.RunJob(job)
except Exception as err:
alive = False
print err
+ '''
self.builder.queue.task_done()
@@ -763,10 +782,13 @@ class Builder:
Args:
commit_upto: Commit number to use (0..self.count-1)
"""
- commit = self.commits[commit_upto]
- subject = commit.subject.translate(trans_valid_chars)
- commit_dir = ('%02d_of_%02d_g%s_%s' % (commit_upto + 1,
- self.commit_count, commit.hash, subject[:20]))
+ if self.commits:
+ commit = self.commits[commit_upto]
+ subject = commit.subject.translate(trans_valid_chars)
+ commit_dir = ('%02d_of_%02d_g%s_%s' % (commit_upto + 1,
+ self.commit_count, commit.hash, subject[:20]))
+ else:
+ commit_dir = 'current'
output_dir = os.path.join(self.base_dir, commit_dir)
return output_dir
@@ -1308,14 +1330,18 @@ class Builder:
show_detail: Show detail for each board
show_bloat: Show detail for each function
"""
- self.commit_count = len(commits)
+ self.commit_count = len(commits) if commits else 1
self.commits = commits
self.ResetResultSummary(board_selected)
for commit_upto in range(0, self.commit_count, self._step):
board_dict, err_lines = self.GetResultSummary(board_selected,
commit_upto, read_func_sizes=show_bloat)
- msg = '%02d: %s' % (commit_upto + 1, commits[commit_upto].subject)
+ if commits:
+ msg = '%02d: %s' % (commit_upto + 1,
+ commits[commit_upto].subject)
+ else:
+ msg = 'current'
print self.col.Color(self.col.BLUE, msg)
self.PrintResultSummary(board_selected, board_dict,
err_lines if show_errors else [], show_sizes, show_detail,
@@ -1330,7 +1356,7 @@ class Builder:
commits: Selected commits to build
"""
# First work out how many commits we will build
- count = (len(commits) + self._step - 1) / self._step
+ count = (self.commit_count + self._step - 1) / self._step
self.count = len(board_selected) * count
self.upto = self.warned = self.fail = 0
self._timestamps = collections.deque()
@@ -1377,13 +1403,14 @@ class Builder:
"""
return os.path.join(self._working_dir, '%02d' % thread_num)
- def _PrepareThread(self, thread_num):
+ def _PrepareThread(self, thread_num, setup_git):
"""Prepare the working directory for a thread.
This clones or fetches the repo into the thread's work directory.
Args:
thread_num: Thread number (0, 1, ...)
+ setup_git: True to set up a git repo clone
"""
thread_dir = self.GetThreadDir(thread_num)
Mkdir(thread_dir)
@@ -1392,7 +1419,7 @@ class Builder:
# Clone the repo if it doesn't already exist
# TODO(sjg at chromium): Perhaps some git hackery to symlink instead, so
# we have a private index but uses the origin repo's contents?
- if self.git_dir:
+ if setup_git and self.git_dir:
src_dir = os.path.abspath(self.git_dir)
if os.path.exists(git_dir):
gitutil.Fetch(git_dir, thread_dir)
@@ -1400,17 +1427,18 @@ class Builder:
print 'Cloning repo for thread %d' % thread_num
gitutil.Clone(src_dir, thread_dir)
- def _PrepareWorkingSpace(self, max_threads):
+ def _PrepareWorkingSpace(self, max_threads, setup_git):
"""Prepare the working directory for use.
Set up the git repo for each thread.
Args:
max_threads: Maximum number of threads we expect to need.
+ setup_git: True to set up a git repo clone
"""
Mkdir(self._working_dir)
for thread in range(max_threads):
- self._PrepareThread(thread)
+ self._PrepareThread(thread, setup_git)
def _PrepareOutputSpace(self):
"""Get the output directories ready to receive files.
@@ -1437,12 +1465,13 @@ class Builder:
show_errors: True to show summarised error/warning info
keep_outputs: True to save build output files
"""
- self.commit_count = len(commits)
+ self.commit_count = len(commits) if commits else 1
self.commits = commits
self.ResetResultSummary(board_selected)
Mkdir(self.base_dir)
- self._PrepareWorkingSpace(min(self.num_threads, len(board_selected)))
+ self._PrepareWorkingSpace(min(self.num_threads, len(board_selected)),
+ commits is not None)
self._PrepareOutputSpace()
self.SetupBuild(board_selected, commits)
self.ProcessResult(None)
diff --git a/tools/buildman/control.py b/tools/buildman/control.py
index 75b6498..12a9699 100644
--- a/tools/buildman/control.py
+++ b/tools/buildman/control.py
@@ -21,15 +21,20 @@ def GetPlural(count):
"""Returns a plural 's' if count is not 1"""
return 's' if count != 1 else ''
-def GetActionSummary(is_summary, count, selected, options):
+def GetActionSummary(is_summary, commits, selected, options):
"""Return a string summarising the intended action.
Returns:
Summary string.
"""
- count = (count + options.step - 1) / options.step
- str = '%s %d commit%s for %d boards' % (
- 'Summary of' if is_summary else 'Building', count, GetPlural(count),
+ if commits:
+ count = len(commits)
+ count = (count + options.step - 1) / options.step
+ commit_str = '%d commit%s' % (count, GetPlural(count))
+ else:
+ commit_str = 'current source'
+ str = '%s %s for %d boards' % (
+ 'Summary of' if is_summary else 'Building', commit_str,
len(selected))
str += ' (%d thread%s, %d job%s per thread)' % (options.threads,
GetPlural(options.threads), options.jobs, GetPlural(options.jobs))
@@ -53,13 +58,18 @@ def ShowActions(series, why_selected, boards_selected, builder, options):
col = terminal.Color()
print 'Dry run, so not doing much. But I would do this:'
print
- print GetActionSummary(False, len(series.commits), boards_selected,
+ if series:
+ commits = series.commits
+ else:
+ commits = None
+ print GetActionSummary(False, commits, boards_selected,
options)
print 'Build directory: %s' % builder.base_dir
- for upto in range(0, len(series.commits), options.step):
- commit = series.commits[upto]
- print ' ', col.Color(col.YELLOW, commit.hash, bright=False),
- print commit.subject
+ if commits:
+ for upto in range(0, len(series.commits), options.step):
+ commit = series.commits[upto]
+ print ' ', col.Color(col.YELLOW, commit.hash, bright=False),
+ print commit.subject
print
for arg in why_selected:
if arg != 'all':
@@ -93,15 +103,16 @@ def DoBuildman(options, args):
count = options.count
if count == -1:
if not options.branch:
- str = 'Please use -b to specify a branch to build'
- print col.Color(col.RED, str)
- sys.exit(1)
- count = gitutil.CountCommitsInBranch(options.git_dir, options.branch)
- if count is None:
- str = "Branch '%s' not found or has no upstream" % options.branch
- print col.Color(col.RED, str)
- sys.exit(1)
- count += 1 # Build upstream commit also
+ count = 1
+ else:
+ count = gitutil.CountCommitsInBranch(options.git_dir,
+ options.branch)
+ if count is None:
+ str = ("Branch '%s' not found or has no upstream" %
+ options.branch)
+ print col.Color(col.RED, str)
+ sys.exit(1)
+ count += 1 # Build upstream commit also
if not count:
str = ("No commits found to process in branch '%s': "
@@ -132,17 +143,21 @@ def DoBuildman(options, args):
# upstream/master~..branch but that isn't possible if upstream/master is
# a merge commit (it will list all the commits that form part of the
# merge)
- range_expr = gitutil.GetRangeInBranch(options.git_dir, options.branch)
- upstream_commit = gitutil.GetUpstream(options.git_dir, options.branch)
- series = patchstream.GetMetaDataForList(upstream_commit, options.git_dir,
- 1)
- # Conflicting tags are not a problem for buildman, since it does not use
- # them. For example, Series-version is not useful for buildman. On the
- # other hand conflicting tags will cause an error. So allow later tags
- # to overwrite earlier ones.
- series.allow_overwrite = True
- series = patchstream.GetMetaDataForList(range_expr, options.git_dir, None,
- series)
+ if options.branch:
+ range_expr = gitutil.GetRangeInBranch(options.git_dir, options.branch)
+ upstream_commit = gitutil.GetUpstream(options.git_dir, options.branch)
+ series = patchstream.GetMetaDataForList(upstream_commit,
+ options.git_dir, 1)
+
+ # Conflicting tags are not a problem for buildman, since it does not
+ # use them. For example, Series-version is not useful for buildman. On
+ # the other hand conflicting tags will cause an error. So allow later
+ # tags to overwrite earlier ones.
+ series.allow_overwrite = True
+ series = patchstream.GetMetaDataForList(range_expr, options.git_dir, None,
+ series)
+ else:
+ series = None
# By default we have one thread per CPU. But if there are not enough jobs
# we can have fewer threads and use a high '-j' value for make.
@@ -162,7 +177,11 @@ def DoBuildman(options, args):
sys.exit(1)
# Create a new builder with the selected options
- output_dir = os.path.join(options.output_dir, options.branch)
+ if options.branch:
+ dirname = options.branch
+ else:
+ dirname = 'current'
+ output_dir = os.path.join(options.output_dir, dirname)
builder = Builder(toolchains, output_dir, options.git_dir,
options.threads, options.jobs, gnu_make=gnu_make, checkout=True,
show_unknown=options.show_unknown, step=options.step)
@@ -180,15 +199,21 @@ def DoBuildman(options, args):
# Work out which boards to build
board_selected = boards.GetSelectedDict()
- print GetActionSummary(options.summary, count, board_selected, options)
+ if series:
+ commits = series.commits
+ else:
+ commits = None
+
+ print GetActionSummary(options.summary, commits, board_selected,
+ options)
if options.summary:
# We can't show function sizes without board details at present
if options.show_bloat:
options.show_detail = True
- builder.ShowSummary(series.commits, board_selected,
+ builder.ShowSummary(commits, board_selected,
options.show_errors, options.show_sizes,
options.show_detail, options.show_bloat)
else:
- builder.BuildBoards(series.commits, board_selected,
+ builder.BuildBoards(commits, board_selected,
options.show_errors, options.keep_outputs)
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 04/20] buildman: Move BuilderThread code to its own file
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (2 preceding siblings ...)
2014-08-09 21:32 ` [U-Boot] [PATCH v6 03/20] buildman: Allow building of current source tree Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 05/20] buildman: Sort command line options Simon Glass
` (15 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
The builder.py file is getting too long, so split out some code.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to fix indentation in teminal.py
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/builder.py | 439 +---------------------------------------
tools/buildman/builderthread.py | 434 +++++++++++++++++++++++++++++++++++++++
2 files changed, 442 insertions(+), 431 deletions(-)
create mode 100644 tools/buildman/builderthread.py
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index d2b72d5..e2da0eb 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -6,7 +6,6 @@
#
import collections
-import errno
from datetime import datetime, timedelta
import glob
import os
@@ -15,9 +14,9 @@ import Queue
import shutil
import string
import sys
-import threading
import time
+import builderthread
import command
import gitutil
import terminal
@@ -97,428 +96,6 @@ OUTCOME_OK, OUTCOME_WARNING, OUTCOME_ERROR, OUTCOME_UNKNOWN = range(4)
trans_valid_chars = string.maketrans("/: ", "---")
-def Mkdir(dirname):
- """Make a directory if it doesn't already exist.
-
- Args:
- dirname: Directory to create
- """
- try:
- os.mkdir(dirname)
- except OSError as err:
- if err.errno == errno.EEXIST:
- pass
- else:
- raise
-
-class BuilderJob:
- """Holds information about a job to be performed by a thread
-
- Members:
- board: Board object to build
- commits: List of commit options to build.
- """
- def __init__(self):
- self.board = None
- self.commits = []
-
-
-class ResultThread(threading.Thread):
- """This thread processes results from builder threads.
-
- It simply passes the results on to the builder. There is only one
- result thread, and this helps to serialise the build output.
- """
- def __init__(self, builder):
- """Set up a new result thread
-
- Args:
- builder: Builder which will be sent each result
- """
- threading.Thread.__init__(self)
- self.builder = builder
-
- def run(self):
- """Called to start up the result thread.
-
- We collect the next result job and pass it on to the build.
- """
- while True:
- result = self.builder.out_queue.get()
- self.builder.ProcessResult(result)
- self.builder.out_queue.task_done()
-
-
-class BuilderThread(threading.Thread):
- """This thread builds U-Boot for a particular board.
-
- An input queue provides each new job. We run 'make' to build U-Boot
- and then pass the results on to the output queue.
-
- Members:
- builder: The builder which contains information we might need
- thread_num: Our thread number (0-n-1), used to decide on a
- temporary directory
- """
- def __init__(self, builder, thread_num):
- """Set up a new builder thread"""
- threading.Thread.__init__(self)
- self.builder = builder
- self.thread_num = thread_num
-
- def Make(self, commit, brd, stage, cwd, *args, **kwargs):
- """Run 'make' on a particular commit and board.
-
- The source code will already be checked out, so the 'commit'
- argument is only for information.
-
- Args:
- commit: Commit object that is being built
- brd: Board object that is being built
- stage: Stage of the build. Valid stages are:
- distclean - can be called to clean source
- config - called to configure for a board
- build - the main make invocation - it does the build
- args: A list of arguments to pass to 'make'
- kwargs: A list of keyword arguments to pass to command.RunPipe()
-
- Returns:
- CommandResult object
- """
- return self.builder.do_make(commit, brd, stage, cwd, *args,
- **kwargs)
-
- def RunCommit(self, commit_upto, brd, work_dir, do_config, force_build,
- force_build_failures):
- """Build a particular commit.
-
- If the build is already done, and we are not forcing a build, we skip
- the build and just return the previously-saved results.
-
- Args:
- commit_upto: Commit number to build (0...n-1)
- brd: Board object to build
- work_dir: Directory to which the source will be checked out
- do_config: True to run a make <board>_defconfig on the source
- force_build: Force a build even if one was previously done
- force_build_failures: Force a bulid if the previous result showed
- failure
-
- Returns:
- tuple containing:
- - CommandResult object containing the results of the build
- - boolean indicating whether 'make config' is still needed
- """
- # Create a default result - it will be overwritte by the call to
- # self.Make() below, in the event that we do a build.
- result = command.CommandResult()
- result.return_code = 0
- if self.builder.in_tree:
- out_dir = work_dir
- else:
- out_dir = os.path.join(work_dir, 'build')
-
- # Check if the job was already completed last time
- done_file = self.builder.GetDoneFile(commit_upto, brd.target)
- result.already_done = os.path.exists(done_file)
- will_build = (force_build or force_build_failures or
- not result.already_done)
- if result.already_done and will_build:
- # Get the return code from that build and use it
- with open(done_file, 'r') as fd:
- result.return_code = int(fd.readline())
- err_file = self.builder.GetErrFile(commit_upto, brd.target)
- if os.path.exists(err_file) and os.stat(err_file).st_size:
- result.stderr = 'bad'
- elif not force_build:
- # The build passed, so no need to build it again
- will_build = False
-
- if will_build:
- # We are going to have to build it. First, get a toolchain
- if not self.toolchain:
- try:
- self.toolchain = self.builder.toolchains.Select(brd.arch)
- except ValueError as err:
- result.return_code = 10
- result.stdout = ''
- result.stderr = str(err)
- # TODO(sjg at chromium.org): This gets swallowed, but needs
- # to be reported.
-
- if self.toolchain:
- # Checkout the right commit
- if self.builder.commits:
- commit = self.builder.commits[commit_upto]
- if self.builder.checkout:
- git_dir = os.path.join(work_dir, '.git')
- gitutil.Checkout(commit.hash, git_dir, work_dir,
- force=True)
- else:
- commit = 'current'
-
- # Set up the environment and command line
- env = self.toolchain.MakeEnvironment()
- Mkdir(out_dir)
- args = []
- cwd = work_dir
- if not self.builder.in_tree:
- if commit_upto is None:
- # In this case we are building in the original source
- # directory (i.e. the current directory where buildman
- # is invoked. The output directory is set to this
- # thread's selected work directory.
- #
- # Symlinks can confuse U-Boot's Makefile since
- # we may use '..' in our path, so remove them.
- work_dir = os.path.realpath(work_dir)
- args.append('O=%s/build' % work_dir)
- cwd = None
- else:
- args.append('O=build')
- args.append('-s')
- if self.builder.num_jobs is not None:
- args.extend(['-j', str(self.builder.num_jobs)])
- config_args = ['%s_defconfig' % brd.target]
- config_out = ''
- args.extend(self.builder.toolchains.GetMakeArguments(brd))
-
- # If we need to reconfigure, do that now
- if do_config:
- result = self.Make(commit, brd, 'distclean', cwd,
- 'distclean', *args, env=env)
- result = self.Make(commit, brd, 'config', cwd,
- *(args + config_args), env=env)
- config_out = result.combined
- do_config = False # No need to configure next time
- if result.return_code == 0:
- result = self.Make(commit, brd, 'build', cwd, *args,
- env=env)
- result.stdout = config_out + result.stdout
- else:
- result.return_code = 1
- result.stderr = 'No tool chain for %s\n' % brd.arch
- result.already_done = False
-
- result.toolchain = self.toolchain
- result.brd = brd
- result.commit_upto = commit_upto
- result.out_dir = out_dir
- return result, do_config
-
- def _WriteResult(self, result, keep_outputs):
- """Write a built result to the output directory.
-
- Args:
- result: CommandResult object containing result to write
- keep_outputs: True to store the output binaries, False
- to delete them
- """
- # Fatal error
- if result.return_code < 0:
- return
-
- # Aborted?
- if result.stderr and 'No child processes' in result.stderr:
- return
-
- if result.already_done:
- return
-
- # Write the output and stderr
- output_dir = self.builder._GetOutputDir(result.commit_upto)
- Mkdir(output_dir)
- build_dir = self.builder.GetBuildDir(result.commit_upto,
- result.brd.target)
- Mkdir(build_dir)
-
- outfile = os.path.join(build_dir, 'log')
- with open(outfile, 'w') as fd:
- if result.stdout:
- fd.write(result.stdout)
-
- errfile = self.builder.GetErrFile(result.commit_upto,
- result.brd.target)
- if result.stderr:
- with open(errfile, 'w') as fd:
- fd.write(result.stderr)
- elif os.path.exists(errfile):
- os.remove(errfile)
-
- if result.toolchain:
- # Write the build result and toolchain information.
- done_file = self.builder.GetDoneFile(result.commit_upto,
- result.brd.target)
- with open(done_file, 'w') as fd:
- fd.write('%s' % result.return_code)
- with open(os.path.join(build_dir, 'toolchain'), 'w') as fd:
- print >>fd, 'gcc', result.toolchain.gcc
- print >>fd, 'path', result.toolchain.path
- print >>fd, 'cross', result.toolchain.cross
- print >>fd, 'arch', result.toolchain.arch
- fd.write('%s' % result.return_code)
-
- with open(os.path.join(build_dir, 'toolchain'), 'w') as fd:
- print >>fd, 'gcc', result.toolchain.gcc
- print >>fd, 'path', result.toolchain.path
-
- # Write out the image and function size information and an objdump
- env = result.toolchain.MakeEnvironment()
- lines = []
- for fname in ['u-boot', 'spl/u-boot-spl']:
- cmd = ['%snm' % self.toolchain.cross, '--size-sort', fname]
- nm_result = command.RunPipe([cmd], capture=True,
- capture_stderr=True, cwd=result.out_dir,
- raise_on_error=False, env=env)
- if nm_result.stdout:
- nm = self.builder.GetFuncSizesFile(result.commit_upto,
- result.brd.target, fname)
- with open(nm, 'w') as fd:
- print >>fd, nm_result.stdout,
-
- cmd = ['%sobjdump' % self.toolchain.cross, '-h', fname]
- dump_result = command.RunPipe([cmd], capture=True,
- capture_stderr=True, cwd=result.out_dir,
- raise_on_error=False, env=env)
- rodata_size = ''
- if dump_result.stdout:
- objdump = self.builder.GetObjdumpFile(result.commit_upto,
- result.brd.target, fname)
- with open(objdump, 'w') as fd:
- print >>fd, dump_result.stdout,
- for line in dump_result.stdout.splitlines():
- fields = line.split()
- if len(fields) > 5 and fields[1] == '.rodata':
- rodata_size = fields[2]
-
- cmd = ['%ssize' % self.toolchain.cross, fname]
- size_result = command.RunPipe([cmd], capture=True,
- capture_stderr=True, cwd=result.out_dir,
- raise_on_error=False, env=env)
- if size_result.stdout:
- lines.append(size_result.stdout.splitlines()[1] + ' ' +
- rodata_size)
-
- # Write out the image sizes file. This is similar to the output
- # of binutil's 'size' utility, but it omits the header line and
- # adds an additional hex value at the end of each line for the
- # rodata size
- if len(lines):
- sizes = self.builder.GetSizesFile(result.commit_upto,
- result.brd.target)
- with open(sizes, 'w') as fd:
- print >>fd, '\n'.join(lines)
-
- # Now write the actual build output
- if keep_outputs:
- patterns = ['u-boot', '*.bin', 'u-boot.dtb', '*.map',
- 'include/autoconf.mk', 'spl/u-boot-spl',
- 'spl/u-boot-spl.bin']
- for pattern in patterns:
- file_list = glob.glob(os.path.join(result.out_dir, pattern))
- for fname in file_list:
- shutil.copy(fname, build_dir)
-
-
- def RunJob(self, job):
- """Run a single job
-
- A job consists of a building a list of commits for a particular board.
-
- Args:
- job: Job to build
- """
- brd = job.board
- work_dir = self.builder.GetThreadDir(self.thread_num)
- self.toolchain = None
- if job.commits:
- # Run 'make board_defconfig' on the first commit
- do_config = True
- commit_upto = 0
- force_build = False
- for commit_upto in range(0, len(job.commits), job.step):
- result, request_config = self.RunCommit(commit_upto, brd,
- work_dir, do_config,
- force_build or self.builder.force_build,
- self.builder.force_build_failures)
- failed = result.return_code or result.stderr
- did_config = do_config
- if failed and not do_config:
- # If our incremental build failed, try building again
- # with a reconfig.
- if self.builder.force_config_on_failure:
- result, request_config = self.RunCommit(commit_upto,
- brd, work_dir, True, True, False)
- did_config = True
- if not self.builder.force_reconfig:
- do_config = request_config
-
- # If we built that commit, then config is done. But if we got
- # an warning, reconfig next time to force it to build the same
- # files that created warnings this time. Otherwise an
- # incremental build may not build the same file, and we will
- # think that the warning has gone away.
- # We could avoid this by using -Werror everywhere...
- # For errors, the problem doesn't happen, since presumably
- # the build stopped and didn't generate output, so will retry
- # that file next time. So we could detect warnings and deal
- # with them specially here. For now, we just reconfigure if
- # anything goes work.
- # Of course this is substantially slower if there are build
- # errors/warnings (e.g. 2-3x slower even if only 10% of builds
- # have problems).
- if (failed and not result.already_done and not did_config and
- self.builder.force_config_on_failure):
- # If this build failed, try the next one with a
- # reconfigure.
- # Sometimes if the board_config.h file changes it can mess
- # with dependencies, and we get:
- # make: *** No rule to make target `include/autoconf.mk',
- # needed by `depend'.
- do_config = True
- force_build = True
- else:
- force_build = False
- if self.builder.force_config_on_failure:
- if failed:
- do_config = True
- result.commit_upto = commit_upto
- if result.return_code < 0:
- raise ValueError('Interrupt')
-
- # We have the build results, so output the result
- self._WriteResult(result, job.keep_outputs)
- self.builder.out_queue.put(result)
- else:
- # Just build the currently checked-out build
- result, request_config = self.RunCommit(None, brd, work_dir, True,
- True, self.builder.force_build_failures)
- result.commit_upto = 0
- self._WriteResult(result, job.keep_outputs)
- self.builder.out_queue.put(result)
-
- def run(self):
- """Our thread's run function
-
- This thread picks a job from the queue, runs it, and then goes to the
- next job.
- """
- alive = True
- while True:
- job = self.builder.queue.get()
- if self.builder.active and alive:
- self.RunJob(job)
- '''
- try:
- if self.builder.active and alive:
- self.RunJob(job)
- except Exception as err:
- alive = False
- print err
- '''
- self.builder.queue.task_done()
-
-
class Builder:
"""Class for building U-Boot for a particular commit.
@@ -639,13 +216,13 @@ class Builder:
self.queue = Queue.Queue()
self.out_queue = Queue.Queue()
for i in range(self.num_threads):
- t = BuilderThread(self, i)
+ t = builderthread.BuilderThread(self, i)
t.setDaemon(True)
t.start()
self.threads.append(t)
self.last_line_len = 0
- t = ResultThread(self)
+ t = builderthread.ResultThread(self)
t.setDaemon(True)
t.start()
self.threads.append(t)
@@ -1385,7 +962,7 @@ class Builder:
for self.commit_upto in range(self.commit_count):
self.SelectCommit(commits[self.commit_upto])
self.SelectOutputDir()
- Mkdir(self.output_dir)
+ builderthread.Mkdir(self.output_dir)
self.BuildBoardsForCommit(board_selected, keep_outputs)
board_dict, err_lines = self.GetResultSummary()
@@ -1413,7 +990,7 @@ class Builder:
setup_git: True to set up a git repo clone
"""
thread_dir = self.GetThreadDir(thread_num)
- Mkdir(thread_dir)
+ builderthread.Mkdir(thread_dir)
git_dir = os.path.join(thread_dir, '.git')
# Clone the repo if it doesn't already exist
@@ -1436,7 +1013,7 @@ class Builder:
max_threads: Maximum number of threads we expect to need.
setup_git: True to set up a git repo clone
"""
- Mkdir(self._working_dir)
+ builderthread.Mkdir(self._working_dir)
for thread in range(max_threads):
self._PrepareThread(thread, setup_git)
@@ -1469,7 +1046,7 @@ class Builder:
self.commits = commits
self.ResetResultSummary(board_selected)
- Mkdir(self.base_dir)
+ builderthread.Mkdir(self.base_dir)
self._PrepareWorkingSpace(min(self.num_threads, len(board_selected)),
commits is not None)
self._PrepareOutputSpace()
@@ -1478,7 +1055,7 @@ class Builder:
# Create jobs to build all commits for each board
for brd in board_selected.itervalues():
- job = BuilderJob()
+ job = builderthread.BuilderJob()
job.board = brd
job.commits = commits
job.keep_outputs = keep_outputs
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py
new file mode 100644
index 0000000..32297e2
--- /dev/null
+++ b/tools/buildman/builderthread.py
@@ -0,0 +1,434 @@
+# Copyright (c) 2014 Google, Inc
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+
+import errno
+import glob
+import os
+import shutil
+import threading
+
+import command
+import gitutil
+
+def Mkdir(dirname):
+ """Make a directory if it doesn't already exist.
+
+ Args:
+ dirname: Directory to create
+ """
+ try:
+ os.mkdir(dirname)
+ except OSError as err:
+ if err.errno == errno.EEXIST:
+ pass
+ else:
+ raise
+
+class BuilderJob:
+ """Holds information about a job to be performed by a thread
+
+ Members:
+ board: Board object to build
+ commits: List of commit options to build.
+ """
+ def __init__(self):
+ self.board = None
+ self.commits = []
+
+
+class ResultThread(threading.Thread):
+ """This thread processes results from builder threads.
+
+ It simply passes the results on to the builder. There is only one
+ result thread, and this helps to serialise the build output.
+ """
+ def __init__(self, builder):
+ """Set up a new result thread
+
+ Args:
+ builder: Builder which will be sent each result
+ """
+ threading.Thread.__init__(self)
+ self.builder = builder
+
+ def run(self):
+ """Called to start up the result thread.
+
+ We collect the next result job and pass it on to the build.
+ """
+ while True:
+ result = self.builder.out_queue.get()
+ self.builder.ProcessResult(result)
+ self.builder.out_queue.task_done()
+
+
+class BuilderThread(threading.Thread):
+ """This thread builds U-Boot for a particular board.
+
+ An input queue provides each new job. We run 'make' to build U-Boot
+ and then pass the results on to the output queue.
+
+ Members:
+ builder: The builder which contains information we might need
+ thread_num: Our thread number (0-n-1), used to decide on a
+ temporary directory
+ """
+ def __init__(self, builder, thread_num):
+ """Set up a new builder thread"""
+ threading.Thread.__init__(self)
+ self.builder = builder
+ self.thread_num = thread_num
+
+ def Make(self, commit, brd, stage, cwd, *args, **kwargs):
+ """Run 'make' on a particular commit and board.
+
+ The source code will already be checked out, so the 'commit'
+ argument is only for information.
+
+ Args:
+ commit: Commit object that is being built
+ brd: Board object that is being built
+ stage: Stage of the build. Valid stages are:
+ distclean - can be called to clean source
+ config - called to configure for a board
+ build - the main make invocation - it does the build
+ args: A list of arguments to pass to 'make'
+ kwargs: A list of keyword arguments to pass to command.RunPipe()
+
+ Returns:
+ CommandResult object
+ """
+ return self.builder.do_make(commit, brd, stage, cwd, *args,
+ **kwargs)
+
+ def RunCommit(self, commit_upto, brd, work_dir, do_config, force_build,
+ force_build_failures):
+ """Build a particular commit.
+
+ If the build is already done, and we are not forcing a build, we skip
+ the build and just return the previously-saved results.
+
+ Args:
+ commit_upto: Commit number to build (0...n-1)
+ brd: Board object to build
+ work_dir: Directory to which the source will be checked out
+ do_config: True to run a make <board>_defconfig on the source
+ force_build: Force a build even if one was previously done
+ force_build_failures: Force a bulid if the previous result showed
+ failure
+
+ Returns:
+ tuple containing:
+ - CommandResult object containing the results of the build
+ - boolean indicating whether 'make config' is still needed
+ """
+ # Create a default result - it will be overwritte by the call to
+ # self.Make() below, in the event that we do a build.
+ result = command.CommandResult()
+ result.return_code = 0
+ if self.builder.in_tree:
+ out_dir = work_dir
+ else:
+ out_dir = os.path.join(work_dir, 'build')
+
+ # Check if the job was already completed last time
+ done_file = self.builder.GetDoneFile(commit_upto, brd.target)
+ result.already_done = os.path.exists(done_file)
+ will_build = (force_build or force_build_failures or
+ not result.already_done)
+ if result.already_done and will_build:
+ # Get the return code from that build and use it
+ with open(done_file, 'r') as fd:
+ result.return_code = int(fd.readline())
+ err_file = self.builder.GetErrFile(commit_upto, brd.target)
+ if os.path.exists(err_file) and os.stat(err_file).st_size:
+ result.stderr = 'bad'
+ elif not force_build:
+ # The build passed, so no need to build it again
+ will_build = False
+
+ if will_build:
+ # We are going to have to build it. First, get a toolchain
+ if not self.toolchain:
+ try:
+ self.toolchain = self.builder.toolchains.Select(brd.arch)
+ except ValueError as err:
+ result.return_code = 10
+ result.stdout = ''
+ result.stderr = str(err)
+ # TODO(sjg at chromium.org): This gets swallowed, but needs
+ # to be reported.
+
+ if self.toolchain:
+ # Checkout the right commit
+ if self.builder.commits:
+ commit = self.builder.commits[commit_upto]
+ if self.builder.checkout:
+ git_dir = os.path.join(work_dir, '.git')
+ gitutil.Checkout(commit.hash, git_dir, work_dir,
+ force=True)
+ else:
+ commit = 'current'
+
+ # Set up the environment and command line
+ env = self.toolchain.MakeEnvironment()
+ Mkdir(out_dir)
+ args = []
+ cwd = work_dir
+ if not self.builder.in_tree:
+ if commit_upto is None:
+ # In this case we are building in the original source
+ # directory (i.e. the current directory where buildman
+ # is invoked. The output directory is set to this
+ # thread's selected work directory.
+ #
+ # Symlinks can confuse U-Boot's Makefile since
+ # we may use '..' in our path, so remove them.
+ work_dir = os.path.realpath(work_dir)
+ args.append('O=%s/build' % work_dir)
+ cwd = None
+ else:
+ args.append('O=build')
+ args.append('-s')
+ if self.builder.num_jobs is not None:
+ args.extend(['-j', str(self.builder.num_jobs)])
+ config_args = ['%s_defconfig' % brd.target]
+ config_out = ''
+ args.extend(self.builder.toolchains.GetMakeArguments(brd))
+
+ # If we need to reconfigure, do that now
+ if do_config:
+ result = self.Make(commit, brd, 'distclean', cwd,
+ 'distclean', *args, env=env)
+ result = self.Make(commit, brd, 'config', cwd,
+ *(args + config_args), env=env)
+ config_out = result.combined
+ do_config = False # No need to configure next time
+ if result.return_code == 0:
+ result = self.Make(commit, brd, 'build', cwd, *args,
+ env=env)
+ result.stdout = config_out + result.stdout
+ else:
+ result.return_code = 1
+ result.stderr = 'No tool chain for %s\n' % brd.arch
+ result.already_done = False
+
+ result.toolchain = self.toolchain
+ result.brd = brd
+ result.commit_upto = commit_upto
+ result.out_dir = out_dir
+ return result, do_config
+
+ def _WriteResult(self, result, keep_outputs):
+ """Write a built result to the output directory.
+
+ Args:
+ result: CommandResult object containing result to write
+ keep_outputs: True to store the output binaries, False
+ to delete them
+ """
+ # Fatal error
+ if result.return_code < 0:
+ return
+
+ # Aborted?
+ if result.stderr and 'No child processes' in result.stderr:
+ return
+
+ if result.already_done:
+ return
+
+ # Write the output and stderr
+ output_dir = self.builder._GetOutputDir(result.commit_upto)
+ Mkdir(output_dir)
+ build_dir = self.builder.GetBuildDir(result.commit_upto,
+ result.brd.target)
+ Mkdir(build_dir)
+
+ outfile = os.path.join(build_dir, 'log')
+ with open(outfile, 'w') as fd:
+ if result.stdout:
+ fd.write(result.stdout)
+
+ errfile = self.builder.GetErrFile(result.commit_upto,
+ result.brd.target)
+ if result.stderr:
+ with open(errfile, 'w') as fd:
+ fd.write(result.stderr)
+ elif os.path.exists(errfile):
+ os.remove(errfile)
+
+ if result.toolchain:
+ # Write the build result and toolchain information.
+ done_file = self.builder.GetDoneFile(result.commit_upto,
+ result.brd.target)
+ with open(done_file, 'w') as fd:
+ fd.write('%s' % result.return_code)
+ with open(os.path.join(build_dir, 'toolchain'), 'w') as fd:
+ print >>fd, 'gcc', result.toolchain.gcc
+ print >>fd, 'path', result.toolchain.path
+ print >>fd, 'cross', result.toolchain.cross
+ print >>fd, 'arch', result.toolchain.arch
+ fd.write('%s' % result.return_code)
+
+ with open(os.path.join(build_dir, 'toolchain'), 'w') as fd:
+ print >>fd, 'gcc', result.toolchain.gcc
+ print >>fd, 'path', result.toolchain.path
+
+ # Write out the image and function size information and an objdump
+ env = result.toolchain.MakeEnvironment()
+ lines = []
+ for fname in ['u-boot', 'spl/u-boot-spl']:
+ cmd = ['%snm' % self.toolchain.cross, '--size-sort', fname]
+ nm_result = command.RunPipe([cmd], capture=True,
+ capture_stderr=True, cwd=result.out_dir,
+ raise_on_error=False, env=env)
+ if nm_result.stdout:
+ nm = self.builder.GetFuncSizesFile(result.commit_upto,
+ result.brd.target, fname)
+ with open(nm, 'w') as fd:
+ print >>fd, nm_result.stdout,
+
+ cmd = ['%sobjdump' % self.toolchain.cross, '-h', fname]
+ dump_result = command.RunPipe([cmd], capture=True,
+ capture_stderr=True, cwd=result.out_dir,
+ raise_on_error=False, env=env)
+ rodata_size = ''
+ if dump_result.stdout:
+ objdump = self.builder.GetObjdumpFile(result.commit_upto,
+ result.brd.target, fname)
+ with open(objdump, 'w') as fd:
+ print >>fd, dump_result.stdout,
+ for line in dump_result.stdout.splitlines():
+ fields = line.split()
+ if len(fields) > 5 and fields[1] == '.rodata':
+ rodata_size = fields[2]
+
+ cmd = ['%ssize' % self.toolchain.cross, fname]
+ size_result = command.RunPipe([cmd], capture=True,
+ capture_stderr=True, cwd=result.out_dir,
+ raise_on_error=False, env=env)
+ if size_result.stdout:
+ lines.append(size_result.stdout.splitlines()[1] + ' ' +
+ rodata_size)
+
+ # Write out the image sizes file. This is similar to the output
+ # of binutil's 'size' utility, but it omits the header line and
+ # adds an additional hex value at the end of each line for the
+ # rodata size
+ if len(lines):
+ sizes = self.builder.GetSizesFile(result.commit_upto,
+ result.brd.target)
+ with open(sizes, 'w') as fd:
+ print >>fd, '\n'.join(lines)
+
+ # Now write the actual build output
+ if keep_outputs:
+ patterns = ['u-boot', '*.bin', 'u-boot.dtb', '*.map',
+ 'include/autoconf.mk', 'spl/u-boot-spl',
+ 'spl/u-boot-spl.bin']
+ for pattern in patterns:
+ file_list = glob.glob(os.path.join(result.out_dir, pattern))
+ for fname in file_list:
+ shutil.copy(fname, build_dir)
+
+
+ def RunJob(self, job):
+ """Run a single job
+
+ A job consists of a building a list of commits for a particular board.
+
+ Args:
+ job: Job to build
+ """
+ brd = job.board
+ work_dir = self.builder.GetThreadDir(self.thread_num)
+ self.toolchain = None
+ if job.commits:
+ # Run 'make board_defconfig' on the first commit
+ do_config = True
+ commit_upto = 0
+ force_build = False
+ for commit_upto in range(0, len(job.commits), job.step):
+ result, request_config = self.RunCommit(commit_upto, brd,
+ work_dir, do_config,
+ force_build or self.builder.force_build,
+ self.builder.force_build_failures)
+ failed = result.return_code or result.stderr
+ did_config = do_config
+ if failed and not do_config:
+ # If our incremental build failed, try building again
+ # with a reconfig.
+ if self.builder.force_config_on_failure:
+ result, request_config = self.RunCommit(commit_upto,
+ brd, work_dir, True, True, False)
+ did_config = True
+ if not self.builder.force_reconfig:
+ do_config = request_config
+
+ # If we built that commit, then config is done. But if we got
+ # an warning, reconfig next time to force it to build the same
+ # files that created warnings this time. Otherwise an
+ # incremental build may not build the same file, and we will
+ # think that the warning has gone away.
+ # We could avoid this by using -Werror everywhere...
+ # For errors, the problem doesn't happen, since presumably
+ # the build stopped and didn't generate output, so will retry
+ # that file next time. So we could detect warnings and deal
+ # with them specially here. For now, we just reconfigure if
+ # anything goes work.
+ # Of course this is substantially slower if there are build
+ # errors/warnings (e.g. 2-3x slower even if only 10% of builds
+ # have problems).
+ if (failed and not result.already_done and not did_config and
+ self.builder.force_config_on_failure):
+ # If this build failed, try the next one with a
+ # reconfigure.
+ # Sometimes if the board_config.h file changes it can mess
+ # with dependencies, and we get:
+ # make: *** No rule to make target `include/autoconf.mk',
+ # needed by `depend'.
+ do_config = True
+ force_build = True
+ else:
+ force_build = False
+ if self.builder.force_config_on_failure:
+ if failed:
+ do_config = True
+ result.commit_upto = commit_upto
+ if result.return_code < 0:
+ raise ValueError('Interrupt')
+
+ # We have the build results, so output the result
+ self._WriteResult(result, job.keep_outputs)
+ self.builder.out_queue.put(result)
+ else:
+ # Just build the currently checked-out build
+ result, request_config = self.RunCommit(None, brd, work_dir, True,
+ True, self.builder.force_build_failures)
+ result.commit_upto = 0
+ self._WriteResult(result, job.keep_outputs)
+ self.builder.out_queue.put(result)
+
+ def run(self):
+ """Our thread's run function
+
+ This thread picks a job from the queue, runs it, and then goes to the
+ next job.
+ """
+ alive = True
+ while True:
+ job = self.builder.queue.get()
+ if self.builder.active and alive:
+ self.RunJob(job)
+ '''
+ try:
+ if self.builder.active and alive:
+ self.RunJob(job)
+ except Exception as err:
+ alive = False
+ print err
+ '''
+ self.builder.queue.task_done()
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 05/20] buildman: Sort command line options
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (3 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 04/20] buildman: Move BuilderThread code to its own file Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 06/20] buildman: Refactor output options Simon Glass
` (14 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
These options have got slightly out of order. Fix them.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to fix the spelling of 'colour'
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/buildman.py | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/tools/buildman/buildman.py b/tools/buildman/buildman.py
index 42847ac..da6025a 100755
--- a/tools/buildman/buildman.py
+++ b/tools/buildman/buildman.py
@@ -70,6 +70,9 @@ parser.add_option('-c', '--count', dest='count', type='int',
parser.add_option('-C', '--force-reconfig', dest='force_reconfig',
action='store_true', default=False,
help='Reconfigure for every commit (disable incremental build)')
+parser.add_option('-d', '--detail', dest='show_detail',
+ action='store_true', default=False,
+ help='Show detailed information for each board in summary')
parser.add_option('-e', '--show_errors', action='store_true',
default=False, help='Show errors and warnings')
parser.add_option('-f', '--force-build', dest='force_build',
@@ -78,9 +81,6 @@ parser.add_option('-f', '--force-build', dest='force_build',
parser.add_option('-F', '--force-build-failures', dest='force_build_failures',
action='store_true', default=False,
help='Force build of previously-failed build')
-parser.add_option('-d', '--detail', dest='show_detail',
- action='store_true', default=False,
- help='Show detailed information for each board in summary')
parser.add_option('-g', '--git', type='string',
help='Git repo containing branch to build', default='.')
parser.add_option('-H', '--full-help', action='store_true', dest='full_help',
@@ -96,6 +96,9 @@ parser.add_option('--list-tool-chains', action='store_true', default=False,
help='List available tool chains')
parser.add_option('-n', '--dry-run', action='store_true', dest='dry_run',
default=False, help="Do a try run (describe actions, but no nothing)")
+parser.add_option('-o', '--output-dir', type='string',
+ dest='output_dir', default='..',
+ help='Directory where all builds happen and buildman has its workspace (default is ../)')
parser.add_option('-Q', '--quick', action='store_true',
default=False, help='Do a rough build, with limited warning resolution')
parser.add_option('-s', '--summary', action='store_true',
@@ -110,9 +113,6 @@ parser.add_option('-T', '--threads', type='int',
default=None, help='Number of builder threads to use')
parser.add_option('-u', '--show_unknown', action='store_true',
default=False, help='Show boards with unknown build result')
-parser.add_option('-o', '--output-dir', type='string',
- dest='output_dir', default='..',
- help='Directory where all builds happen and buildman has its workspace (default is ../)')
parser.usage = """buildman -b <branch> [options]
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 06/20] buildman: Refactor output options
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (4 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 05/20] buildman: Sort command line options Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 07/20] buildman: Add verbose option to display errors as they happen Simon Glass
` (13 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
We need the output options to be available in several places. It's a pain
to pass them into each function. Make them properties of the builder and
add a single function to set them up. At the same time, add a function which
produces summary output using these options.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to avoid using --no-decorate when not available
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/builder.py | 46 +++++++++++++++++++++++++++-------------------
tools/buildman/control.py | 8 ++++----
tools/buildman/test.py | 6 +++---
3 files changed, 34 insertions(+), 26 deletions(-)
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index e2da0eb..a6c43e0 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -235,6 +235,20 @@ class Builder:
for t in self.threads:
del t
+ def SetDisplayOptions(self, show_errors=False, show_sizes=False,
+ show_detail=False, show_bloat=False):
+ """Setup display options for the builder.
+
+ show_errors: True to show summarised error/warning info
+ show_sizes: Show size deltas
+ show_detail: Show detail for each board
+ show_bloat: Show detail for each function
+ """
+ self._show_errors = show_errors
+ self._show_sizes = show_sizes
+ self._show_detail = show_detail
+ self._show_bloat = show_bloat
+
def _AddTimestamp(self):
"""Add a new timestamp to the list and record the build period.
@@ -891,9 +905,18 @@ class Builder:
print "Boards not built (%d): %s" % (len(not_built),
', '.join(not_built))
+ def ProduceResultSummary(self, commit_upto, commits, board_selected):
+ board_dict, err_lines = self.GetResultSummary(board_selected,
+ commit_upto, read_func_sizes=self._show_bloat)
+ if commits:
+ msg = '%02d: %s' % (commit_upto + 1,
+ commits[commit_upto].subject)
+ print self.col.Color(self.col.BLUE, msg)
+ self.PrintResultSummary(board_selected, board_dict,
+ err_lines if self._show_errors else [],
+ self._show_sizes, self._show_detail, self._show_bloat)
- def ShowSummary(self, commits, board_selected, show_errors, show_sizes,
- show_detail, show_bloat):
+ def ShowSummary(self, commits, board_selected):
"""Show a build summary for U-Boot for a given board list.
Reset the result summary, then repeatedly call GetResultSummary on
@@ -902,27 +925,13 @@ class Builder:
Args:
commit: Commit objects to summarise
board_selected: Dict containing boards to summarise
- show_errors: Show errors that occured
- show_sizes: Show size deltas
- show_detail: Show detail for each board
- show_bloat: Show detail for each function
"""
self.commit_count = len(commits) if commits else 1
self.commits = commits
self.ResetResultSummary(board_selected)
for commit_upto in range(0, self.commit_count, self._step):
- board_dict, err_lines = self.GetResultSummary(board_selected,
- commit_upto, read_func_sizes=show_bloat)
- if commits:
- msg = '%02d: %s' % (commit_upto + 1,
- commits[commit_upto].subject)
- else:
- msg = 'current'
- print self.col.Color(self.col.BLUE, msg)
- self.PrintResultSummary(board_selected, board_dict,
- err_lines if show_errors else [], show_sizes, show_detail,
- show_bloat)
+ self.ProduceResultSummary(commit_upto, commits, board_selected)
def SetupBuild(self, board_selected, commits):
@@ -1032,14 +1041,13 @@ class Builder:
if dirname not in dir_list:
shutil.rmtree(dirname)
- def BuildBoards(self, commits, board_selected, show_errors, keep_outputs):
+ def BuildBoards(self, commits, board_selected, keep_outputs):
"""Build all commits for a list of boards
Args:
commits: List of commits to be build, each a Commit object
boards_selected: Dict of selected boards, key is target name,
value is Board object
- show_errors: True to show summarised error/warning info
keep_outputs: True to save build output files
"""
self.commit_count = len(commits) if commits else 1
diff --git a/tools/buildman/control.py b/tools/buildman/control.py
index 12a9699..d8fa74b 100644
--- a/tools/buildman/control.py
+++ b/tools/buildman/control.py
@@ -207,13 +207,13 @@ def DoBuildman(options, args):
print GetActionSummary(options.summary, commits, board_selected,
options)
+ builder.SetDisplayOptions(options.show_errors, options.show_sizes,
+ options.show_detail, options.show_bloat)
if options.summary:
# We can't show function sizes without board details at present
if options.show_bloat:
options.show_detail = True
- builder.ShowSummary(commits, board_selected,
- options.show_errors, options.show_sizes,
- options.show_detail, options.show_bloat)
+ builder.ShowSummary(commits, board_selected)
else:
builder.BuildBoards(commits, board_selected,
- options.show_errors, options.keep_outputs)
+ options.keep_outputs)
diff --git a/tools/buildman/test.py b/tools/buildman/test.py
index 068784a..c6923af 100644
--- a/tools/buildman/test.py
+++ b/tools/buildman/test.py
@@ -137,9 +137,9 @@ class TestBuild(unittest.TestCase):
board_selected = self.boards.GetSelectedDict()
#build.BuildCommits(self.commits, board_selected, False)
- build.BuildBoards(self.commits, board_selected, False, False)
- build.ShowSummary(self.commits, board_selected, True, False,
- False, False)
+ build.BuildBoards(self.commits, board_selected, False)
+ build.SetDisplayOptions(show_errors=True);
+ build.ShowSummary(self.commits, board_selected)
def _testGit(self):
"""Test basic builder operation by building a branch"""
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 07/20] buildman: Add verbose option to display errors as they happen
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (5 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 06/20] buildman: Refactor output options Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 08/20] buildman: Remove unused non-incremental build method code Simon Glass
` (12 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
Normally buildman operates in two passes - one to do the build and another
to summarise the errors. Add a verbose option (-v) to display build problems
as they happen. With -e also given, this will display errors too.
When building the current source tree (rather than a list of commits in a
branch), both -v and -e are enabled automatically.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to move 'git log' command into a function
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/README | 46 ++++++++++++++++++++++++++++++++++++----------
tools/buildman/builder.py | 16 +++++++++++++---
tools/buildman/buildman.py | 2 ++
tools/buildman/control.py | 4 +++-
tools/buildman/test.py | 3 ++-
5 files changed, 56 insertions(+), 15 deletions(-)
diff --git a/tools/buildman/README b/tools/buildman/README
index 1c919af..6ba24c0 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -41,9 +41,10 @@ Theory of Operation
Buildman is a builder. It is not make, although it runs make. It does not
produce any useful output on the terminal while building, except for
-progress information. All the output (errors, warnings and binaries if you
-are ask for them) is stored in output directories, which you can look at
-while the build is progressing, or when it is finished.
+progress information (except with -v, see below). All the output (errors,
+warnings and binaries if you are ask for them) is stored in output
+directories, which you can look at while the build is progressing, or when
+it is finished.
Buildman produces a concise summary of which boards succeeded and failed.
It shows which commit introduced which board failure using a simple
@@ -77,12 +78,17 @@ Buildman automatically selects the correct tool chain for each board. You
must supply suitable tool chains, but buildman takes care of selecting the
right one.
-Buildman always builds a branch, and always builds the upstream commit as
-well, for comparison. It cannot build individual commits at present, unless
-(maybe) you point it at an empty branch. Put all your commits in a branch,
-set the branch's upstream to a valid value, and all will be well. Otherwise
-buildman will perform random actions. Use -n to check what the random
-actions might be.
+Buildman generally builds a branch (with the -b flag), and in this case
+builds the upstream commit as well, for comparison. It cannot build
+individual commits at present, unless (maybe) you point it at an empty
+branch. Put all your commits in a branch, set the branch's upstream to a
+valid value, and all will be well. Otherwise buildman will perform random
+actions. Use -n to check what the random actions might be.
+
+If you just want to build the current source tree, leave off the -b flag.
+This will display results and errors as they happen. You can still look
+at them later using -s. Note that buildman will assume that the source
+has changed, and will build all specified boards in this case.
Buildman is optimised for building many commits at once, for many boards.
On multi-core machines, Buildman is fast because it uses most of the
@@ -659,6 +665,15 @@ It is expected that any variables added are dealt with in U-Boot's
config.mk file and documented in the README.
+Quick Sanity Check
+==================
+
+If you have made changes and want to do a quick sanity check of the
+currently-checked-out source, run buildman without the -b flag. This will
+build the selected boards and display build status and errors as it runs
+(i.e. -v amd -e are enabled automatically).
+
+
Other options
=============
@@ -685,7 +700,15 @@ First you need to set up your tool chains - see the 'Setting up' section
for details. Once you have your required toolchain(s) detected then you are
ready to go.
-Buildman works on entire branches, so the normal use is:
+To build the current source tree, run buildman without a -b flag:
+
+ ./tools/buildman/buildman <list of things to build>
+
+This will build the current source tree for the given boards and display
+the results and errors.
+
+However buildman usually works on entire branches, and for that you must
+specify a board flag:
./tools/buildman/buildman -b <branch_name> <list of things to build>
@@ -698,6 +721,9 @@ buildman just shows a summary, with red indicating that a commit introduced
an error and green indicating that a commit fixed an error. Use the -e
flag to see the full errors.
+If you really want to see build results as they happen, use -v when doing a
+build (and -e if you want to see errors as well).
+
You don't need to stick around on that branch while buildman is running. It
checks out its own copy of the source code, so you can change branches,
add commits, etc. without affecting the build in progress.
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index a6c43e0..dd7f5a4 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -321,7 +321,8 @@ class Builder:
"""Process the result of a build, showing progress information
Args:
- result: A CommandResult object
+ result: A CommandResult object, which indicates the result for
+ a single build
"""
col = terminal.Color()
if result:
@@ -339,6 +340,13 @@ class Builder:
self.warned += 1
if result.already_done:
self.already_done += 1
+ if self._verbose:
+ print '\r',
+ self.ClearLine(0)
+ boards_selected = {target : result.brd}
+ self.ResetResultSummary(boards_selected)
+ self.ProduceResultSummary(result.commit_upto, self.commits,
+ boards_selected)
else:
target = '(starting)'
@@ -362,7 +370,7 @@ class Builder:
name += target
print line + name,
- length = 13 + len(name)
+ length = 14 + len(name)
self.ClearLine(length)
def _GetOutputDir(self, commit_upto):
@@ -1041,7 +1049,7 @@ class Builder:
if dirname not in dir_list:
shutil.rmtree(dirname)
- def BuildBoards(self, commits, board_selected, keep_outputs):
+ def BuildBoards(self, commits, board_selected, keep_outputs, verbose):
"""Build all commits for a list of boards
Args:
@@ -1049,9 +1057,11 @@ class Builder:
boards_selected: Dict of selected boards, key is target name,
value is Board object
keep_outputs: True to save build output files
+ verbose: Display build results as they are completed
"""
self.commit_count = len(commits) if commits else 1
self.commits = commits
+ self._verbose = verbose
self.ResetResultSummary(board_selected)
builderthread.Mkdir(self.base_dir)
diff --git a/tools/buildman/buildman.py b/tools/buildman/buildman.py
index da6025a..f32557f 100755
--- a/tools/buildman/buildman.py
+++ b/tools/buildman/buildman.py
@@ -113,6 +113,8 @@ parser.add_option('-T', '--threads', type='int',
default=None, help='Number of builder threads to use')
parser.add_option('-u', '--show_unknown', action='store_true',
default=False, help='Show boards with unknown build result')
+parser.add_option('-v', '--verbose', action='store_true',
+ default=False, help='Show build results while the build progresses')
parser.usage = """buildman -b <branch> [options]
diff --git a/tools/buildman/control.py b/tools/buildman/control.py
index d8fa74b..cc8593f 100644
--- a/tools/buildman/control.py
+++ b/tools/buildman/control.py
@@ -158,6 +158,8 @@ def DoBuildman(options, args):
series)
else:
series = None
+ options.verbose = True
+ options.show_errors = True
# By default we have one thread per CPU. But if there are not enough jobs
# we can have fewer threads and use a high '-j' value for make.
@@ -216,4 +218,4 @@ def DoBuildman(options, args):
builder.ShowSummary(commits, board_selected)
else:
builder.BuildBoards(commits, board_selected,
- options.keep_outputs)
+ options.keep_outputs, options.verbose)
diff --git a/tools/buildman/test.py b/tools/buildman/test.py
index c6923af..a724d83 100644
--- a/tools/buildman/test.py
+++ b/tools/buildman/test.py
@@ -137,7 +137,8 @@ class TestBuild(unittest.TestCase):
board_selected = self.boards.GetSelectedDict()
#build.BuildCommits(self.commits, board_selected, False)
- build.BuildBoards(self.commits, board_selected, False)
+ build.BuildBoards(self.commits, board_selected, keep_outputs=False,
+ verbose=False)
build.SetDisplayOptions(show_errors=True);
build.ShowSummary(self.commits, board_selected)
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 08/20] buildman: Remove unused non-incremental build method code
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (6 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 07/20] buildman: Add verbose option to display errors as they happen Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 09/20] buildman: Add an option to specify the buildman config file Simon Glass
` (11 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
The non-incremental build method is no longer used, so remove it.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6:
- Add new patch to implement -c option
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/builder.py | 34 ----------------------------------
tools/buildman/test.py | 1 -
2 files changed, 35 deletions(-)
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index dd7f5a4..22a24b1 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -955,40 +955,6 @@ class Builder:
self.upto = self.warned = self.fail = 0
self._timestamps = collections.deque()
- def BuildBoardsForCommit(self, board_selected, keep_outputs):
- """Build all boards for a single commit"""
- self.SetupBuild(board_selected)
- self.count = len(board_selected)
- for brd in board_selected.itervalues():
- job = BuilderJob()
- job.board = brd
- job.commits = None
- job.keep_outputs = keep_outputs
- self.queue.put(brd)
-
- self.queue.join()
- self.out_queue.join()
- print
- self.ClearLine(0)
-
- def BuildCommits(self, commits, board_selected, show_errors, keep_outputs):
- """Build all boards for all commits (non-incremental)"""
- self.commit_count = len(commits)
-
- self.ResetResultSummary(board_selected)
- for self.commit_upto in range(self.commit_count):
- self.SelectCommit(commits[self.commit_upto])
- self.SelectOutputDir()
- builderthread.Mkdir(self.output_dir)
-
- self.BuildBoardsForCommit(board_selected, keep_outputs)
- board_dict, err_lines = self.GetResultSummary()
- self.PrintResultSummary(board_selected, board_dict,
- err_lines if show_errors else [])
-
- if self.already_done:
- print '%d builds already done' % self.already_done
-
def GetThreadDir(self, thread_num):
"""Get the directory path to the working dir for a thread.
diff --git a/tools/buildman/test.py b/tools/buildman/test.py
index a724d83..502c9b4 100644
--- a/tools/buildman/test.py
+++ b/tools/buildman/test.py
@@ -136,7 +136,6 @@ class TestBuild(unittest.TestCase):
build.do_make = self.Make
board_selected = self.boards.GetSelectedDict()
- #build.BuildCommits(self.commits, board_selected, False)
build.BuildBoards(self.commits, board_selected, keep_outputs=False,
verbose=False)
build.SetDisplayOptions(show_errors=True);
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 09/20] buildman: Add an option to specify the buildman config file
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (7 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 08/20] buildman: Remove unused non-incremental build method code Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 10/20] buildman: Add a message indicating there are no errors Simon Glass
` (10 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
Add a new --config-file option (-G) to specify a different configuration
file from the default ~/.buildman.
Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5:
- Add new patch to add an 'and' operator for board selection
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/buildman/buildman.py | 2 ++
tools/buildman/control.py | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/tools/buildman/buildman.py b/tools/buildman/buildman.py
index f32557f..6ca8dc6 100755
--- a/tools/buildman/buildman.py
+++ b/tools/buildman/buildman.py
@@ -83,6 +83,8 @@ parser.add_option('-F', '--force-build-failures', dest='force_build_failures',
help='Force build of previously-failed build')
parser.add_option('-g', '--git', type='string',
help='Git repo containing branch to build', default='.')
+parser.add_option('-G', '--config-file', type='string',
+ help='Path to buildman config file', default='')
parser.add_option('-H', '--full-help', action='store_true', dest='full_help',
default=False, help='Display the README file')
parser.add_option('-i', '--in-tree', dest='in_tree',
diff --git a/tools/buildman/control.py b/tools/buildman/control.py
index cc8593f..0c9e2cb 100644
--- a/tools/buildman/control.py
+++ b/tools/buildman/control.py
@@ -86,7 +86,7 @@ def DoBuildman(options, args):
"""
gitutil.Setup()
- bsettings.Setup()
+ bsettings.Setup(options.config_file)
options.git_dir = os.path.join(options.git, '.git')
toolchains = toolchain.Toolchains()
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 10/20] buildman: Add a message indicating there are no errors
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (8 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 09/20] buildman: Add an option to specify the buildman config file Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 11/20] buildman: Add a few more toolchain examples to the README Simon Glass
` (9 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
If buildman finds no problems it prints nothing. This can be a bit confusing,
so add a message that all is well.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4:
- Add new patch to add a few more toolchain examples to the README
Changes in v3: None
Changes in v2: None
tools/buildman/builder.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index 22a24b1..d5b8454 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -210,6 +210,7 @@ class Builder:
self.force_reconfig = False
self._step = step
self.in_tree = False
+ self._error_lines = 0
self.col = terminal.Color()
@@ -891,10 +892,13 @@ class Builder:
self.col.MAGENTA)
for arch, target_list in arch_list.iteritems():
print '%10s: %s' % (arch, target_list)
+ self._error_lines += 1
if better_err:
print self.col.Color(self.col.GREEN, '\n'.join(better_err))
+ self._error_lines += 1
if worse_err:
print self.col.Color(self.col.RED, '\n'.join(worse_err))
+ self._error_lines += 1
if show_sizes:
self.PrintSizeSummary(board_selected, board_dict, show_detail,
@@ -937,9 +941,12 @@ class Builder:
self.commit_count = len(commits) if commits else 1
self.commits = commits
self.ResetResultSummary(board_selected)
+ self._error_lines = 0
for commit_upto in range(0, self.commit_count, self._step):
self.ProduceResultSummary(commit_upto, commits, board_selected)
+ if not self._error_lines:
+ print self.col.Color(self.col.GREEN, '(no errors to report)')
def SetupBuild(self, board_selected, commits):
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 11/20] buildman: Add a few more toolchain examples to the README
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (9 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 10/20] buildman: Add a message indicating there are no errors Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 12/20] buildman: Introduce an 'and' operator for board selection Simon Glass
` (8 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
The current README is a bit sparse in this area, so add a few more
examples.
Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4:
- Add new patch to show a message when there are no errors
Changes in v3: None
Changes in v2: None
tools/buildman/README | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/buildman/README b/tools/buildman/README
index 6ba24c0..aaf0a10 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -134,6 +134,8 @@ example:
root: /
rest: /toolchains/*
eldk: /opt/eldk-4.2
+arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
+aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux
[toolchain-alias]
x86: i386
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 12/20] buildman: Introduce an 'and' operator for board selection
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (10 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 11/20] buildman: Add a few more toolchain examples to the README Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-11 17:51 ` York Sun
2014-08-09 21:33 ` [U-Boot] [PATCH v6 13/20] buildman: Allow selection of the number of commits to build Simon Glass
` (7 subsequent siblings)
19 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
Currently buildman allows a list of boards to build to be specified on the
command line. The list can include specific board names, architecture, SOC
and so on.
At present the list of boards is dealt with in an 'OR' fashion, and there
is no way to specify something like 'arm & freescale', meaning boards with
ARM architecture but only those made by Freescale. This would exclude the
PowerPC boards made by Freescale.
Support an '&' operator on the command line to permit this. Ensure that
arguments can be specified in a single string to permit easy shell quoting.
Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4:
- Add new patch to allow the config file to be specified
Changes in v3: None
Changes in v2: None
tools/buildman/README | 9 +++
tools/buildman/board.py | 144 ++++++++++++++++++++++++++++++++++++++++++------
tools/buildman/test.py | 48 ++++++++++++++++
3 files changed, 184 insertions(+), 17 deletions(-)
diff --git a/tools/buildman/README b/tools/buildman/README
index aaf0a10..d4e8404 100644
--- a/tools/buildman/README
+++ b/tools/buildman/README
@@ -108,6 +108,15 @@ behaviour is a superset of exact or substring matching. Examples are:
* '^tegra[23]0$' All boards with either Tegra20 or Tegra30 SoC
* 'powerpc' All PowerPC boards
+While the default is to OR the terms together, you can also make use of
+the '&' operator to limit the selection:
+
+* 'freescale & arm sandbox' All Freescale boards with ARM architecture,
+ plus sandbox
+
+It is convenient to use the -n option to see whaat will be built based on
+the subset given.
+
Buildman does not store intermediate object files. It optionally copies
the binary output into a directory when a build is successful. Size
information is always recorded. It needs a fair bit of disk space to work,
diff --git a/tools/buildman/board.py b/tools/buildman/board.py
index 7bcc932..a333287 100644
--- a/tools/buildman/board.py
+++ b/tools/buildman/board.py
@@ -5,6 +5,72 @@
import re
+class Expr:
+ """A single regular expression for matching boards to build"""
+
+ def __init__(self, expr):
+ """Set up a new Expr object.
+
+ Args:
+ expr: String cotaining regular expression to store
+ """
+ self._expr = expr
+ self._re = re.compile(expr)
+
+ def Matches(self, props):
+ """Check if any of the properties match the regular expression.
+
+ Args:
+ props: List of properties to check
+ Returns:
+ True if any of the properties match the regular expression
+ """
+ for prop in props:
+ if self._re.match(prop):
+ return True
+ return False
+
+ def __str__(self):
+ return self._expr
+
+class Term:
+ """A list of expressions each of which must match with properties.
+
+ This provides a list of 'AND' expressions, meaning that each must
+ match the board properties for that board to be built.
+ """
+ def __init__(self):
+ self._expr_list = []
+ self._board_count = 0
+
+ def AddExpr(self, expr):
+ """Add an Expr object to the list to check.
+
+ Args:
+ expr: New Expr object to add to the list of those that must
+ match for a board to be built.
+ """
+ self._expr_list.append(Expr(expr))
+
+ def __str__(self):
+ """Return some sort of useful string describing the term"""
+ return '&'.join([str(expr) for expr in self._expr_list])
+
+ def Matches(self, props):
+ """Check if any of the properties match this term
+
+ Each of the expressions in the term is checked. All must match.
+
+ Args:
+ props: List of properties to check
+ Returns:
+ True if all of the expressions in the Term match, else False
+ """
+ for expr in self._expr_list:
+ if not expr.Matches(props):
+ return False
+ return True
+
class Board:
"""A particular board that we can build"""
def __init__(self, status, arch, cpu, soc, vendor, board_name, target, options):
@@ -124,6 +190,55 @@ class Boards:
"""
return [board.target for board in self._boards if board.build_it]
+ def _BuildTerms(self, args):
+ """Convert command line arguments to a list of terms.
+
+ This deals with parsing of the arguments. It handles the '&'
+ operator, which joins several expressions into a single Term.
+
+ For example:
+ ['arm & freescale sandbox', 'tegra']
+
+ will produce 3 Terms containing expressions as follows:
+ arm, freescale
+ sandbox
+ tegra
+
+ The first Term has two expressions, both of which must match for
+ a board to be selected.
+
+ Args:
+ args: List of command line arguments
+ Returns:
+ A list of Term objects
+ """
+ syms = []
+ for arg in args:
+ for word in arg.split():
+ sym_build = []
+ for term in word.split('&'):
+ if term:
+ sym_build.append(term)
+ sym_build.append('&')
+ syms += sym_build[:-1]
+ terms = []
+ term = None
+ oper = None
+ for sym in syms:
+ if sym == '&':
+ oper = sym
+ elif oper:
+ term.AddExpr(sym)
+ oper = None
+ else:
+ if term:
+ terms.append(term)
+ term = Term()
+ term.AddExpr(sym)
+ if term:
+ terms.append(term)
+ return terms
+
def SelectBoards(self, args):
"""Mark boards selected based on args
@@ -137,26 +252,21 @@ class Boards:
due to each argument, arranged by argument.
"""
result = {}
- argres = {}
- for arg in args:
- result[arg] = 0
- argres[arg] = re.compile(arg)
+ terms = self._BuildTerms(args)
+
result['all'] = 0
+ for term in terms:
+ result[str(term)] = 0
for board in self._boards:
- if args:
- for arg in args:
- argre = argres[arg]
- match = False
- for prop in board.props:
- match = argre.match(prop)
- if match:
- break
- if match:
- if not board.build_it:
- board.build_it = True
- result[arg] += 1
- result['all'] += 1
+ if terms:
+ match = False
+ for term in terms:
+ if term.Matches(board.props):
+ board.build_it = True
+ result[str(term)] += 1
+ result['all'] += 1
+ break
else:
board.build_it = True
result['all'] += 1
diff --git a/tools/buildman/test.py b/tools/buildman/test.py
index 502c9b4..a51c942 100644
--- a/tools/buildman/test.py
+++ b/tools/buildman/test.py
@@ -165,5 +165,53 @@ class TestBuild(unittest.TestCase):
args = ['tegra20']
control.DoBuildman(options, args)
+ def testBoardSingle(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['sandbox']),
+ {'all': 1, 'sandbox': 1})
+
+ def testBoardArch(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['arm']),
+ {'all': 2, 'arm': 2})
+
+ def testBoardArchSingle(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['arm sandbox']),
+ {'all': 3, 'arm': 2, 'sandbox' : 1})
+
+ def testBoardArchSingleMultiWord(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['arm', 'sandbox']),
+ {'all': 3, 'arm': 2, 'sandbox' : 1})
+
+ def testBoardSingleAnd(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['Tester & arm']),
+ {'all': 2, 'Tester&arm': 2})
+
+ def testBoardTwoAnd(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['Tester', '&', 'arm',
+ 'Tester' '&', 'powerpc',
+ 'sandbox']),
+ {'all': 5, 'Tester&powerpc': 2, 'Tester&arm': 2,
+ 'sandbox' : 1})
+
+ def testBoardAll(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards([]), {'all': 5})
+
+ def testBoardRegularExpression(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['T.*r&^Po']),
+ {'T.*r&^Po': 2, 'all': 2})
+
+ def testBoardDuplicate(self):
+ """Test single board selection"""
+ self.assertEqual(self.boards.SelectBoards(['sandbox sandbox',
+ 'sandbox']),
+ {'all': 1, 'sandbox': 1})
+
if __name__ == "__main__":
unittest.main()
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 12/20] buildman: Introduce an 'and' operator for board selection
2014-08-09 21:33 ` [U-Boot] [PATCH v6 12/20] buildman: Introduce an 'and' operator for board selection Simon Glass
@ 2014-08-11 17:51 ` York Sun
0 siblings, 0 replies; 34+ messages in thread
From: York Sun @ 2014-08-11 17:51 UTC (permalink / raw)
To: u-boot
On 08/09/2014 02:33 PM, Simon Glass wrote:
> Currently buildman allows a list of boards to build to be specified on the
> command line. The list can include specific board names, architecture, SOC
> and so on.
>
> At present the list of boards is dealt with in an 'OR' fashion, and there
> is no way to specify something like 'arm & freescale', meaning boards with
> ARM architecture but only those made by Freescale. This would exclude the
> PowerPC boards made by Freescale.
>
> Support an '&' operator on the command line to permit this. Ensure that
> arguments can be specified in a single string to permit easy shell quoting.
>
> Suggested-by: York Sun <yorksun@freescale.com>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
Acked-by: York Sun <yorksun@freescale.com>
York
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 13/20] buildman: Allow selection of the number of commits to build
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (11 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 12/20] buildman: Introduce an 'and' operator for board selection Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-11 17:51 ` York Sun
2014-08-09 21:33 ` [U-Boot] [PATCH v6 14/20] patman: Move the 'git log' command into a function Simon Glass
` (6 subsequent siblings)
19 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
It is useful to be able to build only some of the commits in a branch. Add
support for the -c option to allow this. It was previously parsed by
buildman but not implemented.
Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4:
- Add new patch to remove unused non-incremental build method code
Changes in v3: None
Changes in v2: None
tools/buildman/control.py | 31 +++++++++++++++++++------------
1 file changed, 19 insertions(+), 12 deletions(-)
diff --git a/tools/buildman/control.py b/tools/buildman/control.py
index 0c9e2cb..98a07a2 100644
--- a/tools/buildman/control.py
+++ b/tools/buildman/control.py
@@ -144,18 +144,25 @@ def DoBuildman(options, args):
# a merge commit (it will list all the commits that form part of the
# merge)
if options.branch:
- range_expr = gitutil.GetRangeInBranch(options.git_dir, options.branch)
- upstream_commit = gitutil.GetUpstream(options.git_dir, options.branch)
- series = patchstream.GetMetaDataForList(upstream_commit,
- options.git_dir, 1)
-
- # Conflicting tags are not a problem for buildman, since it does not
- # use them. For example, Series-version is not useful for buildman. On
- # the other hand conflicting tags will cause an error. So allow later
- # tags to overwrite earlier ones.
- series.allow_overwrite = True
- series = patchstream.GetMetaDataForList(range_expr, options.git_dir, None,
- series)
+ if count == -1:
+ range_expr = gitutil.GetRangeInBranch(options.git_dir,
+ options.branch)
+ upstream_commit = gitutil.GetUpstream(options.git_dir,
+ options.branch)
+ series = patchstream.GetMetaDataForList(upstream_commit,
+ options.git_dir, 1)
+
+ # Conflicting tags are not a problem for buildman, since it does
+ # not use them. For example, Series-version is not useful for
+ # buildman. On the other hand conflicting tags will cause an
+ # error. So allow later tags to overwrite earlier ones.
+ series.allow_overwrite = True
+ series = patchstream.GetMetaDataForList(range_expr,
+ options.git_dir, None, series)
+ else:
+ # Honour the count
+ series = patchstream.GetMetaDataForList(options.branch,
+ options.git_dir, count)
else:
series = None
options.verbose = True
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 14/20] patman: Move the 'git log' command into a function
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (12 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 13/20] buildman: Allow selection of the number of commits to build Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 15/20] patman: Only use git's --no-decorate when available Simon Glass
` (5 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
Move the code that builds a 'git log' command into a function so we can more
easily adjust it.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5:
- Update test for verbose option
Changes in v4: None
Changes in v3:
- Add new patch to add a verbose option
Changes in v2: None
tools/patman/gitutil.py | 33 ++++++++++++++++++++++++++++-----
tools/patman/patchstream.py | 11 +++--------
2 files changed, 31 insertions(+), 13 deletions(-)
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 65754f5..a8b0a1a 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -14,6 +14,31 @@ import terminal
import checkpatch
import settings
+def LogCmd(commit_range, git_dir=None, oneline=False, reverse=False,
+ count=None):
+ """Create a command to perform a 'git log'
+
+ Args:
+ commit_range: Range expression to use for log, None for none
+ git_dir: Path to git repositiory (None to use default)
+ oneline: True to use --oneline, else False
+ reverse: True to reverse the log (--reverse)
+ count: Number of commits to list, or None for no limit
+ Return:
+ List containing command and arguments to run
+ """
+ cmd = ['git']
+ if git_dir:
+ cmd += ['--git-dir', git_dir]
+ cmd += ['log', '--no-color']
+ if oneline:
+ cmd.append('--oneline')
+ cmd.append('--no-decorate')
+ if count is not None:
+ cmd.append('-n%d' % count)
+ if commit_range:
+ cmd.append(commit_range)
+ return cmd
def CountCommitsToBranch():
"""Returns number of commits between HEAD and the tracking branch.
@@ -24,8 +49,7 @@ def CountCommitsToBranch():
Return:
Number of patches that exist on top of the branch
"""
- pipe = [['git', 'log', '--no-color', '--oneline', '--no-decorate',
- '@{upstream}..'],
+ pipe = [LogCmd('@{upstream}..', oneline=True),
['wc', '-l']]
stdout = command.RunPipe(pipe, capture=True, oneline=True).stdout
patch_count = int(stdout)
@@ -87,8 +111,7 @@ def CountCommitsInBranch(git_dir, branch, include_upstream=False):
range_expr = GetRangeInBranch(git_dir, branch, include_upstream)
if not range_expr:
return None
- pipe = [['git', '--git-dir', git_dir, 'log', '--oneline', '--no-decorate',
- range_expr],
+ pipe = [LogCmd(range_expr, git_dir=git_dir, oneline=True),
['wc', '-l']]
result = command.RunPipe(pipe, capture=True, oneline=True)
patch_count = int(result.stdout)
@@ -102,7 +125,7 @@ def CountCommits(commit_range):
Return:
Number of patches that exist on top of the branch
"""
- pipe = [['git', 'log', '--oneline', '--no-decorate', commit_range],
+ pipe = [LogCmd(commit_range, oneline=True),
['wc', '-l']]
stdout = command.RunPipe(pipe, capture=True, oneline=True).stdout
patch_count = int(stdout)
diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index 3228719..0040468 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -379,14 +379,9 @@ def GetMetaDataForList(commit_range, git_dir=None, count=None,
Returns:
A Series object containing information about the commits.
"""
- params = ['git', 'log', '--no-color', '--reverse', '--no-decorate',
- commit_range]
- if count is not None:
- params[2:2] = ['-n%d' % count]
- if git_dir:
- params[1:1] = ['--git-dir', git_dir]
- pipe = [params]
- stdout = command.RunPipe(pipe, capture=True).stdout
+ params = gitutil.LogCmd(commit_range,reverse=True, count=count,
+ git_dir=git_dir)
+ stdout = command.RunPipe([params], capture=True).stdout
ps = PatchStream(series, is_log=True)
for line in stdout.splitlines():
ps.ProcessLine(line)
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 15/20] patman: Only use git's --no-decorate when available
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (13 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 14/20] patman: Move the 'git log' command into a function Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour' Simon Glass
` (4 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
Older versions of git (e.g. Ubuntu 10.04) do not support this flag. By
default they do not decorate. So only enable this flag when supported.
Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5:
- Update test for output options
Changes in v4: None
Changes in v3:
- Add new patch to refactor output options
Changes in v2: None
tools/patman/gitutil.py | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index a8b0a1a..735c8dd 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -14,6 +14,9 @@ import terminal
import checkpatch
import settings
+# True to use --no-decorate - we check this in Setup()
+use_no_decorate = True
+
def LogCmd(commit_range, git_dir=None, oneline=False, reverse=False,
count=None):
"""Create a command to perform a 'git log'
@@ -33,7 +36,8 @@ def LogCmd(commit_range, git_dir=None, oneline=False, reverse=False,
cmd += ['log', '--no-color']
if oneline:
cmd.append('--oneline')
- cmd.append('--no-decorate')
+ if use_no_decorate:
+ cmd.append('--no-decorate')
if count is not None:
cmd.append('-n%d' % count)
if commit_range:
@@ -566,6 +570,9 @@ def Setup():
alias_fname = GetAliasFile()
if alias_fname:
settings.ReadGitAliases(alias_fname)
+ cmd = LogCmd(None, count=0)
+ use_no_decorate = (command.RunPipe([cmd], raise_on_error=False)
+ .return_code == 0)
def GetHead():
"""Get the hash of the current HEAD
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (14 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 15/20] patman: Only use git's --no-decorate when available Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 22:01 ` Fabio Estevam
2014-08-09 21:33 ` [U-Boot] [PATCH v6 17/20] patman: Fix indentation in terminal.py Simon Glass
` (3 subsequent siblings)
19 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
This code came from another project. Correct the spelling now that it is
in U-Boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4:
- Fix typo in commit message
Changes in v3:
- Add new patch to sort command line options
Changes in v2: None
tools/buildman/builder.py | 46 +++++++++++++++++++++---------------------
tools/buildman/control.py | 12 +++++------
tools/patman/checkpatch.py | 18 ++++++++---------
tools/patman/gitutil.py | 22 ++++++++++----------
tools/patman/patman.py | 4 ++--
tools/patman/series.py | 14 ++++++-------
tools/patman/terminal.py | 50 +++++++++++++++++++++++-----------------------
7 files changed, 83 insertions(+), 83 deletions(-)
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py
index d5b8454..a65d1c3 100644
--- a/tools/buildman/builder.py
+++ b/tools/buildman/builder.py
@@ -105,7 +105,7 @@ class Builder:
base_dir: Base directory to use for builder
checkout: True to check out source, False to skip that step.
This is used for testing.
- col: terminal.Color() object
+ col: terminal.Colour() object
count: Number of commits to build
do_make: Method to call to invoke Make
fail: Number of builds that failed due to error
@@ -212,7 +212,7 @@ class Builder:
self.in_tree = False
self._error_lines = 0
- self.col = terminal.Color()
+ self.col = terminal.Colour()
self.queue = Queue.Queue()
self.out_queue = Queue.Queue()
@@ -325,7 +325,7 @@ class Builder:
result: A CommandResult object, which indicates the result for
a single build
"""
- col = terminal.Color()
+ col = terminal.Colour()
if result:
target = result.brd.target
@@ -353,9 +353,9 @@ class Builder:
# Display separate counts for ok, warned and fail
ok = self.upto - self.warned - self.fail
- line = '\r' + self.col.Color(self.col.GREEN, '%5d' % ok)
- line += self.col.Color(self.col.YELLOW, '%5d' % self.warned)
- line += self.col.Color(self.col.RED, '%5d' % self.fail)
+ line = '\r' + self.col.Colour(self.col.GREEN, '%5d' % ok)
+ line += self.col.Colour(self.col.YELLOW, '%5d' % self.warned)
+ line += self.col.Colour(self.col.RED, '%5d' % self.fail)
name = ' /%-5d ' % self.count
@@ -583,7 +583,7 @@ class Builder:
err_lines_summary.append(err.rstrip())
return board_dict, err_lines_summary
- def AddOutcome(self, board_dict, arch_list, changes, char, color):
+ def AddOutcome(self, board_dict, arch_list, changes, char, colour):
"""Add an output to our list of outcomes for each architecture
This simple function adds failing boards (changes) to the
@@ -595,7 +595,7 @@ class Builder:
arch_list: Dict keyed by arch name. Value is a string containing
a list of board names which failed for that arch.
changes: List of boards to add to arch_list
- color: terminal.Colour object
+ colour: terminal.Colour object
"""
done_arch = {}
for target in changes:
@@ -603,9 +603,9 @@ class Builder:
arch = board_dict[target].arch
else:
arch = 'unknown'
- str = self.col.Color(color, ' ' + target)
+ str = self.col.Colour(colour, ' ' + target)
if not arch in done_arch:
- str = self.col.Color(color, char) + ' ' + str
+ str = self.col.Colour(colour, char) + ' ' + str
done_arch[arch] = True
if not arch in arch_list:
arch_list[arch] = str
@@ -614,10 +614,10 @@ class Builder:
def ColourNum(self, num):
- color = self.col.RED if num > 0 else self.col.GREEN
+ colour = self.col.RED if num > 0 else self.col.GREEN
if num == 0:
return '0'
- return self.col.Color(color, str(num))
+ return self.col.Colour(colour, str(num))
def ResetResultSummary(self, board_selected):
"""Reset the results summary ready for use.
@@ -674,15 +674,15 @@ class Builder:
args = [self.ColourNum(x) for x in args]
indent = ' ' * 15
print ('%s%s: add: %s/%s, grow: %s/%s bytes: %s/%s (%s)' %
- tuple([indent, self.col.Color(self.col.YELLOW, fname)] + args))
+ tuple([indent, self.col.Colour(self.col.YELLOW, fname)] + args))
print '%s %-38s %7s %7s %+7s' % (indent, 'function', 'old', 'new',
'delta')
for diff, name in delta:
if diff:
- color = self.col.RED if diff > 0 else self.col.GREEN
+ colour = self.col.RED if diff > 0 else self.col.GREEN
msg = '%s %-38s %7s %7s %+7d' % (indent, name,
old.get(name, '-'), new.get(name,'-'), diff)
- print self.col.Color(color, msg)
+ print self.col.Colour(colour, msg)
def PrintSizeDetail(self, target_list, show_bloat):
@@ -704,12 +704,12 @@ class Builder:
if name.startswith('_'):
continue
if diff != 0:
- color = self.col.RED if diff > 0 else self.col.GREEN
+ colour = self.col.RED if diff > 0 else self.col.GREEN
msg = ' %s %+d' % (name, diff)
if not printed_target:
print '%10s %-15s:' % ('', result['_target']),
printed_target = True
- print self.col.Color(color, msg),
+ print self.col.Colour(colour, msg),
if printed_target:
print
if show_bloat:
@@ -813,13 +813,13 @@ class Builder:
# Display the average difference in this name for this
# architecture
avg_diff = float(diff) / count
- color = self.col.RED if avg_diff > 0 else self.col.GREEN
+ colour = self.col.RED if avg_diff > 0 else self.col.GREEN
msg = ' %s %+1.1f' % (name, avg_diff)
if not printed_arch:
print '%10s: (for %d/%d boards)' % (arch, count,
arch_count[arch]),
printed_arch = True
- print self.col.Color(color, msg),
+ print self.col.Colour(colour, msg),
if printed_arch:
print
@@ -894,10 +894,10 @@ class Builder:
print '%10s: %s' % (arch, target_list)
self._error_lines += 1
if better_err:
- print self.col.Color(self.col.GREEN, '\n'.join(better_err))
+ print self.col.Colour(self.col.GREEN, '\n'.join(better_err))
self._error_lines += 1
if worse_err:
- print self.col.Color(self.col.RED, '\n'.join(worse_err))
+ print self.col.Colour(self.col.RED, '\n'.join(worse_err))
self._error_lines += 1
if show_sizes:
@@ -923,7 +923,7 @@ class Builder:
if commits:
msg = '%02d: %s' % (commit_upto + 1,
commits[commit_upto].subject)
- print self.col.Color(self.col.BLUE, msg)
+ print self.col.Colour(self.col.BLUE, msg)
self.PrintResultSummary(board_selected, board_dict,
err_lines if self._show_errors else [],
self._show_sizes, self._show_detail, self._show_bloat)
@@ -946,7 +946,7 @@ class Builder:
for commit_upto in range(0, self.commit_count, self._step):
self.ProduceResultSummary(commit_upto, commits, board_selected)
if not self._error_lines:
- print self.col.Color(self.col.GREEN, '(no errors to report)')
+ print self.col.Colour(self.col.GREEN, '(no errors to report)')
def SetupBuild(self, board_selected, commits):
diff --git a/tools/buildman/control.py b/tools/buildman/control.py
index 98a07a2..b11a215 100644
--- a/tools/buildman/control.py
+++ b/tools/buildman/control.py
@@ -55,7 +55,7 @@ def ShowActions(series, why_selected, boards_selected, builder, options):
builder: The builder that will be used to build the commits
options: Command line options object
"""
- col = terminal.Color()
+ col = terminal.Colour()
print 'Dry run, so not doing much. But I would do this:'
print
if series:
@@ -68,7 +68,7 @@ def ShowActions(series, why_selected, boards_selected, builder, options):
if commits:
for upto in range(0, len(series.commits), options.step):
commit = series.commits[upto]
- print ' ', col.Color(col.YELLOW, commit.hash, bright=False),
+ print ' ', col.Colour(col.YELLOW, commit.hash, bright=False),
print commit.subject
print
for arg in why_selected:
@@ -99,7 +99,7 @@ def DoBuildman(options, args):
# Work out how many commits to build. We want to build everything on the
# branch. We also build the upstream commit as a control so we can see
# problems introduced by the first commit on the branch.
- col = terminal.Color()
+ col = terminal.Colour()
count = options.count
if count == -1:
if not options.branch:
@@ -110,14 +110,14 @@ def DoBuildman(options, args):
if count is None:
str = ("Branch '%s' not found or has no upstream" %
options.branch)
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
sys.exit(1)
count += 1 # Build upstream commit also
if not count:
str = ("No commits found to process in branch '%s': "
"set branch's upstream or use -c flag" % options.branch)
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
sys.exit(1)
# Work out what subset of the boards we are building
@@ -135,7 +135,7 @@ def DoBuildman(options, args):
why_selected = boards.SelectBoards(args)
selected = boards.GetSelected()
if not len(selected):
- print col.Color(col.RED, 'No matching boards found')
+ print col.Colour(col.RED, 'No matching boards found')
sys.exit(1)
# Read the metadata from the commits. First look at the upstream commit,
diff --git a/tools/patman/checkpatch.py b/tools/patman/checkpatch.py
index 0d4e935..c5c1603 100644
--- a/tools/patman/checkpatch.py
+++ b/tools/patman/checkpatch.py
@@ -133,17 +133,17 @@ def GetWarningMsg(col, msg_type, fname, line, msg):
msg: Message to report
'''
if msg_type == 'warning':
- msg_type = col.Color(col.YELLOW, msg_type)
+ msg_type = col.Colour(col.YELLOW, msg_type)
elif msg_type == 'error':
- msg_type = col.Color(col.RED, msg_type)
+ msg_type = col.Colour(col.RED, msg_type)
elif msg_type == 'check':
- msg_type = col.Color(col.MAGENTA, msg_type)
+ msg_type = col.Colour(col.MAGENTA, msg_type)
return '%s: %s,%d: %s' % (msg_type, fname, line, msg)
def CheckPatches(verbose, args):
'''Run the checkpatch.pl script on each patch'''
error_count, warning_count, check_count = 0, 0, 0
- col = terminal.Color()
+ col = terminal.Colour()
for fname in args:
result = CheckPatch(fname, verbose)
@@ -152,7 +152,7 @@ def CheckPatches(verbose, args):
warning_count += result.warnings
check_count += result.checks
print '%d errors, %d warnings, %d checks for %s:' % (result.errors,
- result.warnings, result.checks, col.Color(col.BLUE, fname))
+ result.warnings, result.checks, col.Colour(col.BLUE, fname))
if (len(result.problems) != result.errors + result.warnings +
result.checks):
print "Internal error: some problems lost"
@@ -164,11 +164,11 @@ def CheckPatches(verbose, args):
#print stdout
if error_count or warning_count or check_count:
str = 'checkpatch.pl found %d error(s), %d warning(s), %d checks(s)'
- color = col.GREEN
+ colour = col.GREEN
if warning_count:
- color = col.YELLOW
+ colour = col.YELLOW
if error_count:
- color = col.RED
- print col.Color(color, str % (error_count, warning_count, check_count))
+ colour = col.RED
+ print col.Colour(colour, str % (error_count, warning_count, check_count))
return False
return True
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 735c8dd..18133f2 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -221,7 +221,7 @@ def ApplyPatch(verbose, fname):
Args:
fname: filename of patch file to apply
"""
- col = terminal.Color()
+ col = terminal.Colour()
cmd = ['git', 'am', fname]
pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
@@ -247,7 +247,7 @@ def ApplyPatches(verbose, args, start_point):
offset was given.
"""
error_count = 0
- col = terminal.Color()
+ col = terminal.Colour()
# Figure out our current position
cmd = ['git', 'name-rev', 'HEAD', '--name-only']
@@ -255,13 +255,13 @@ def ApplyPatches(verbose, args, start_point):
stdout, stderr = pipe.communicate()
if pipe.returncode:
str = 'Could not find current commit name'
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
print stdout
return False
old_head = stdout.splitlines()[0]
if old_head == 'undefined':
str = "Invalid HEAD '%s'" % stdout.strip()
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
return False
# Checkout the required start point
@@ -271,7 +271,7 @@ def ApplyPatches(verbose, args, start_point):
stdout, stderr = pipe.communicate()
if pipe.returncode:
str = 'Could not move to commit before patch series'
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
print stdout, stderr
return False
@@ -279,7 +279,7 @@ def ApplyPatches(verbose, args, start_point):
for fname in args:
ok, stdout = ApplyPatch(verbose, fname)
if not ok:
- print col.Color(col.RED, 'git am returned errors for %s: will '
+ print col.Colour(col.RED, 'git am returned errors for %s: will '
'skip this patch' % fname)
if verbose:
print stdout
@@ -288,7 +288,7 @@ def ApplyPatches(verbose, args, start_point):
pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE)
stdout, stderr = pipe.communicate()
if pipe.returncode != 0:
- print col.Color(col.RED, 'Unable to skip patch! Aborting...')
+ print col.Colour(col.RED, 'Unable to skip patch! Aborting...')
print stdout
break
@@ -297,7 +297,7 @@ def ApplyPatches(verbose, args, start_point):
pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = pipe.communicate()
if pipe.returncode:
- print col.Color(col.RED, 'Could not move back to head commit')
+ print col.Colour(col.RED, 'Could not move back to head commit')
print stdout, stderr
return error_count == 0
@@ -492,7 +492,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0):
return [lookup_name]
lookup_name = lookup_name.lower()
- col = terminal.Color()
+ col = terminal.Colour()
out_list = []
if level > 10:
@@ -500,7 +500,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0):
if raise_on_error:
raise OSError, msg
else:
- print col.Color(col.RED, msg)
+ print col.Colour(col.RED, msg)
return out_list
if lookup_name:
@@ -509,7 +509,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0):
if raise_on_error:
raise ValueError, msg
else:
- print col.Color(col.RED, msg)
+ print col.Colour(col.RED, msg)
return out_list
for item in alias[lookup_name]:
todo = LookupEmail(item, alias, raise_on_error, level + 1)
diff --git a/tools/patman/patman.py b/tools/patman/patman.py
index c60aa5a..8fdedda 100755
--- a/tools/patman/patman.py
+++ b/tools/patman/patman.py
@@ -119,10 +119,10 @@ else:
# Work out how many patches to send if we can
options.count = gitutil.CountCommitsToBranch() - options.start
- col = terminal.Color()
+ col = terminal.Colour()
if not options.count:
str = 'No commits found to process - please use -c flag'
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
sys.exit(1)
# Read the metadata from the commits
diff --git a/tools/patman/series.py b/tools/patman/series.py
index 88c0d87..40f908a 100644
--- a/tools/patman/series.py
+++ b/tools/patman/series.py
@@ -94,7 +94,7 @@ class Series(dict):
cmd: The git command we would have run
process_tags: Process tags as if they were aliases
"""
- col = terminal.Color()
+ col = terminal.Colour()
print 'Dry run, so not doing much. But I would do this:'
print
print 'Send a total of %d patch%s with %scover letter.' % (
@@ -104,7 +104,7 @@ class Series(dict):
# TODO: Colour the patches according to whether they passed checks
for upto in range(len(args)):
commit = self.commits[upto]
- print col.Color(col.GREEN, ' %s' % args[upto])
+ print col.Colour(col.GREEN, ' %s' % args[upto])
cc_list = list(self._generated_cc[commit.patch])
# Skip items in To list
@@ -116,7 +116,7 @@ class Series(dict):
for email in cc_list:
if email == None:
- email = col.Color(col.YELLOW, "<alias '%s' not found>"
+ email = col.Colour(col.YELLOW, "<alias '%s' not found>"
% tag)
if email:
print ' Cc: ',email
@@ -184,7 +184,7 @@ class Series(dict):
Print an error if something is wrong.
"""
- col = terminal.Color()
+ col = terminal.Colour()
if self.get('version'):
changes_copy = dict(self.changes)
for version in range(1, int(self.version) + 1):
@@ -193,13 +193,13 @@ class Series(dict):
else:
if version > 1:
str = 'Change log missing for v%d' % version
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
for version in changes_copy:
str = 'Change log for unknown version v%d' % version
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
elif self.changes:
str = 'Change log exists, but no version is set'
- print col.Color(col.RED, str)
+ print col.Colour(col.RED, str)
def MakeCcFile(self, process_tags, cover_fname, raise_on_error):
"""Make a cc file for us to use for per-commit Cc automation
diff --git a/tools/patman/terminal.py b/tools/patman/terminal.py
index 597d526..5862aa1 100644
--- a/tools/patman/terminal.py
+++ b/tools/patman/terminal.py
@@ -5,17 +5,17 @@
"""Terminal utilities
-This module handles terminal interaction including ANSI color codes.
+This module handles terminal interaction including ANSI colour codes.
"""
import os
import sys
-# Selection of when we want our output to be colored
-COLOR_IF_TERMINAL, COLOR_ALWAYS, COLOR_NEVER = range(3)
+# Selection of when we want our output to be Coloured
+COLOUR_IF_TERMINAL, COLOUR_ALWAYS, COLOUR_NEVER = range(3)
-class Color(object):
- """Conditionally wraps text in ANSI color escape sequences."""
+class Colour(object):
+ """Conditionally wraps text in ANSI colour escape sequences."""
BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8)
BOLD = -1
BRIGHT_START = '\033[1;%dm'
@@ -23,58 +23,58 @@ class Color(object):
BOLD_START = '\033[1m'
RESET = '\033[0m'
- def __init__(self, colored=COLOR_IF_TERMINAL):
- """Create a new Color object, optionally disabling color output.
+ def __init__(self, coloured=COLOUR_IF_TERMINAL):
+ """Create a new Colour object, optionally disabling colour output.
Args:
- enabled: True if color output should be enabled. If False then this
- class will not add color codes at all.
+ enabled: True if colour output should be enabled. If False then this
+ class will not add colour codes at all.
"""
- self._enabled = (colored == COLOR_ALWAYS or
- (colored == COLOR_IF_TERMINAL and os.isatty(sys.stdout.fileno())))
+ self._enabled = (coloured == COLOUR_ALWAYS or
+ (coloured == COLOUR_IF_TERMINAL and os.isatty(sys.stdout.fileno())))
- def Start(self, color, bright=True):
- """Returns a start color code.
+ def Start(self, colour, bright=True):
+ """Returns a start colour code.
Args:
- color: Color to use, .e.g BLACK, RED, etc.
+ colour: Colour to use, .e.g BLACK, RED, etc.
Returns:
- If color is enabled, returns an ANSI sequence to start the given color,
+ If colour is enabled, returns an ANSI sequence to start the given colour,
otherwise returns empty string
"""
if self._enabled:
base = self.BRIGHT_START if bright else self.NORMAL_START
- return base % (color + 30)
+ return base % (colour + 30)
return ''
def Stop(self):
- """Retruns a stop color code.
+ """Retruns a stop colour code.
Returns:
- If color is enabled, returns an ANSI color reset sequence, otherwise
+ If colour is enabled, returns an ANSI colour reset sequence, otherwise
returns empty string
"""
if self._enabled:
return self.RESET
return ''
- def Color(self, color, text, bright=True):
- """Returns text with conditionally added color escape sequences.
+ def Colour(self, colour, text, bright=True):
+ """Returns text with conditionally added colour escape sequences.
Keyword arguments:
- color: Text color -- one of the color constants defined in this class.
- text: The text to color.
+ colour: Text colour -- one of the colour constants defined in this class.
+ text: The text to colour.
Returns:
If self._enabled is False, returns the original text. If it's True,
- returns text with color escape sequences based on the value of color.
+ returns text with colour escape sequences based on the value of colour.
"""
if not self._enabled:
return text
- if color == self.BOLD:
+ if colour == self.BOLD:
start = self.BOLD_START
else:
base = self.BRIGHT_START if bright else self.NORMAL_START
- start = base % (color + 30)
+ start = base % (colour + 30)
return start + text + self.RESET
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 21:33 ` [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour' Simon Glass
@ 2014-08-09 22:01 ` Fabio Estevam
2014-08-09 22:17 ` Simon Glass
0 siblings, 1 reply; 34+ messages in thread
From: Fabio Estevam @ 2014-08-09 22:01 UTC (permalink / raw)
To: u-boot
Hi Simon,
On Sat, Aug 9, 2014 at 6:33 PM, Simon Glass <sjg@chromium.org> wrote:
> This code came from another project. Correct the spelling now that it is
> in U-Boot.
....
> - col: terminal.Color() object
> + col: terminal.Colour() object
I thought that both forms were correct and that "color" is the
American way and "colour" the British one.
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 22:01 ` Fabio Estevam
@ 2014-08-09 22:17 ` Simon Glass
2014-08-09 22:29 ` Fabio Estevam
0 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-09 22:17 UTC (permalink / raw)
To: u-boot
Hi,
On 9 August 2014 16:01, Fabio Estevam <festevam@gmail.com> wrote:
> Hi Simon,
>
> On Sat, Aug 9, 2014 at 6:33 PM, Simon Glass <sjg@chromium.org> wrote:
>> This code came from another project. Correct the spelling now that it is
>> in U-Boot.
>
> ....
>
>> - col: terminal.Color() object
>> + col: terminal.Colour() object
>
> I thought that both forms were correct and that "color" is the
> American way and "colour" the British one.
Yes, but U-Boot doesn't use American English as I understand it, so
might as well fix it since i'm making other changes in that file.
Regards,
Simon
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 22:17 ` Simon Glass
@ 2014-08-09 22:29 ` Fabio Estevam
2014-08-09 22:34 ` Simon Glass
0 siblings, 1 reply; 34+ messages in thread
From: Fabio Estevam @ 2014-08-09 22:29 UTC (permalink / raw)
To: u-boot
On Sat, Aug 9, 2014 at 7:17 PM, Simon Glass <sjg@chromium.org> wrote:
> Hi,
>
> On 9 August 2014 16:01, Fabio Estevam <festevam@gmail.com> wrote:
>> Hi Simon,
>>
>> On Sat, Aug 9, 2014 at 6:33 PM, Simon Glass <sjg@chromium.org> wrote:
>>> This code came from another project. Correct the spelling now that it is
>>> in U-Boot.
>>
>> ....
>>
>>> - col: terminal.Color() object
>>> + col: terminal.Colour() object
>>
>> I thought that both forms were correct and that "color" is the
>> American way and "colour" the British one.
>
> Yes, but U-Boot doesn't use American English as I understand it, so
I think it just depends on the code's author preference as both
formats are valid.
Doing a quick grep in U-boot source:
$ git grep color | wc -l
646
$ git grep colour | wc -l
43
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 22:29 ` Fabio Estevam
@ 2014-08-09 22:34 ` Simon Glass
2014-08-10 5:23 ` York Sun
2014-08-11 20:34 ` Tom Rini
0 siblings, 2 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 22:34 UTC (permalink / raw)
To: u-boot
Hi Fabio,
On 9 August 2014 16:29, Fabio Estevam <festevam@gmail.com> wrote:
> On Sat, Aug 9, 2014 at 7:17 PM, Simon Glass <sjg@chromium.org> wrote:
>> Hi,
>>
>> On 9 August 2014 16:01, Fabio Estevam <festevam@gmail.com> wrote:
>>> Hi Simon,
>>>
>>> On Sat, Aug 9, 2014 at 6:33 PM, Simon Glass <sjg@chromium.org> wrote:
>>>> This code came from another project. Correct the spelling now that it is
>>>> in U-Boot.
>>>
>>> ....
>>>
>>>> - col: terminal.Color() object
>>>> + col: terminal.Colour() object
>>>
>>> I thought that both forms were correct and that "color" is the
>>> American way and "colour" the British one.
>>
>> Yes, but U-Boot doesn't use American English as I understand it, so
>
> I think it just depends on the code's author preference as both
> formats are valid.
>
> Doing a quick grep in U-boot source:
>
> $ git grep color | wc -l
> 646
> $ git grep colour | wc -l
> 43
Yes that may well be true :-)
Regards,
Simon
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 22:34 ` Simon Glass
@ 2014-08-10 5:23 ` York Sun
2014-08-11 20:34 ` Tom Rini
1 sibling, 0 replies; 34+ messages in thread
From: York Sun @ 2014-08-10 5:23 UTC (permalink / raw)
To: u-boot
On 8/9/14 3:34 PM, "Simon Glass" <sjg@chromium.org> wrote:
>Hi Fabio,
>
>On 9 August 2014 16:29, Fabio Estevam <festevam@gmail.com> wrote:
>> On Sat, Aug 9, 2014 at 7:17 PM, Simon Glass <sjg@chromium.org> wrote:
>>> Hi,
>>>
>>> On 9 August 2014 16:01, Fabio Estevam <festevam@gmail.com> wrote:
>>>> Hi Simon,
>>>>
>>>> On Sat, Aug 9, 2014 at 6:33 PM, Simon Glass <sjg@chromium.org> wrote:
>>>>> This code came from another project. Correct the spelling now that
>>>>>it is
>>>>> in U-Boot.
>>>>
>>>> ....
>>>>
>>>>> - col: terminal.Color() object
>>>>> + col: terminal.Colour() object
>>>>
>>>> I thought that both forms were correct and that "color" is the
>>>> American way and "colour" the British one.
>>>
>>> Yes, but U-Boot doesn't use American English as I understand it, so
>>
>> I think it just depends on the code's author preference as both
>> formats are valid.
>>
>> Doing a quick grep in U-boot source:
>>
>> $ git grep color | wc -l
>> 646
>> $ git grep colour | wc -l
>> 43
>
>Yes that may well be true :-)
I am not bothered by either spelling. But I hate my computer automatically
corrects it.
York
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour'
2014-08-09 22:34 ` Simon Glass
2014-08-10 5:23 ` York Sun
@ 2014-08-11 20:34 ` Tom Rini
1 sibling, 0 replies; 34+ messages in thread
From: Tom Rini @ 2014-08-11 20:34 UTC (permalink / raw)
To: u-boot
On Sat, Aug 09, 2014 at 04:34:47PM -0600, Simon Glass wrote:
> Hi Fabio,
>
> On 9 August 2014 16:29, Fabio Estevam <festevam@gmail.com> wrote:
> > On Sat, Aug 9, 2014 at 7:17 PM, Simon Glass <sjg@chromium.org> wrote:
> >> Hi,
> >>
> >> On 9 August 2014 16:01, Fabio Estevam <festevam@gmail.com> wrote:
> >>> Hi Simon,
> >>>
> >>> On Sat, Aug 9, 2014 at 6:33 PM, Simon Glass <sjg@chromium.org> wrote:
> >>>> This code came from another project. Correct the spelling now that it is
> >>>> in U-Boot.
> >>>
> >>> ....
> >>>
> >>>> - col: terminal.Color() object
> >>>> + col: terminal.Colour() object
> >>>
> >>> I thought that both forms were correct and that "color" is the
> >>> American way and "colour" the British one.
> >>
> >> Yes, but U-Boot doesn't use American English as I understand it, so
> >
> > I think it just depends on the code's author preference as both
> > formats are valid.
> >
> > Doing a quick grep in U-boot source:
> >
> > $ git grep color | wc -l
> > 646
> > $ git grep colour | wc -l
> > 43
>
> Yes that may well be true :-)
... so lets drop this patch, thanks.
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140811/53371119/attachment.pgp>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [U-Boot] [PATCH v6 17/20] patman: Fix indentation in terminal.py
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (15 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 16/20] patman: buildman: Correct spelling of 'colour' Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 18/20] patman: Correct unit tests to run correctly Simon Glass
` (2 subsequent siblings)
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
This code came from a different project with 2-character indentation. Fix
it for U-Boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4:
- Fix missing import in builder.py
Changes in v3:
- Add new patch to move BuilderThread code to its own file
Changes in v2: None
tools/patman/terminal.py | 108 ++++++++++++++++++++++++-----------------------
1 file changed, 55 insertions(+), 53 deletions(-)
diff --git a/tools/patman/terminal.py b/tools/patman/terminal.py
index 5862aa1..be216d3 100644
--- a/tools/patman/terminal.py
+++ b/tools/patman/terminal.py
@@ -15,66 +15,68 @@ import sys
COLOUR_IF_TERMINAL, COLOUR_ALWAYS, COLOUR_NEVER = range(3)
class Colour(object):
- """Conditionally wraps text in ANSI colour escape sequences."""
- BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8)
- BOLD = -1
- BRIGHT_START = '\033[1;%dm'
- NORMAL_START = '\033[22;%dm'
- BOLD_START = '\033[1m'
- RESET = '\033[0m'
+ """Conditionally wraps text in ANSI colour escape sequences."""
+ BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8)
+ BOLD = -1
+ BRIGHT_START = '\033[1;%dm'
+ NORMAL_START = '\033[22;%dm'
+ BOLD_START = '\033[1m'
+ RESET = '\033[0m'
- def __init__(self, coloured=COLOUR_IF_TERMINAL):
- """Create a new Colour object, optionally disabling colour output.
+ def __init__(self, coloured=COLOUR_IF_TERMINAL):
+ """Create a new Colour object, optionally disabling colour output.
- Args:
- enabled: True if colour output should be enabled. If False then this
- class will not add colour codes at all.
- """
- self._enabled = (coloured == COLOUR_ALWAYS or
- (coloured == COLOUR_IF_TERMINAL and os.isatty(sys.stdout.fileno())))
+ Args:
+ enabled: True if colour output should be enabled. If False then this
+ class will not add colour codes at all.
+ """
+ self._enabled = (coloured == COLOUR_ALWAYS or
+ (coloured == COLOUR_IF_TERMINAL and os.isatty(sys.stdout.fileno())))
- def Start(self, colour, bright=True):
- """Returns a start colour code.
+ def Start(self, colour, bright=True):
+ """Returns a start colour code.
- Args:
- colour: Colour to use, .e.g BLACK, RED, etc.
+ Args:
+ colour: Colour to use, .e.g BLACK, RED, etc.
- Returns:
- If colour is enabled, returns an ANSI sequence to start the given colour,
- otherwise returns empty string
- """
- if self._enabled:
- base = self.BRIGHT_START if bright else self.NORMAL_START
- return base % (colour + 30)
- return ''
+ Returns:
+ If colour is enabled, returns an ANSI sequence to start the given
+ colour, otherwise returns empty string
+ """
+ if self._enabled:
+ base = self.BRIGHT_START if bright else self.NORMAL_START
+ return base % (colour + 30)
+ return ''
- def Stop(self):
- """Retruns a stop colour code.
+ def Stop(self):
+ """Retruns a stop colour code.
- Returns:
- If colour is enabled, returns an ANSI colour reset sequence, otherwise
- returns empty string
- """
- if self._enabled:
- return self.RESET
- return ''
+ Returns:
+ If colour is enabled, returns an ANSI colour reset sequence,
+ otherwise returns empty string
+ """
+ if self._enabled:
+ return self.RESET
+ return ''
- def Colour(self, colour, text, bright=True):
- """Returns text with conditionally added colour escape sequences.
+ def Colour(self, colour, text, bright=True):
+ """Returns text with conditionally added colour escape sequences.
- Keyword arguments:
- colour: Text colour -- one of the colour constants defined in this class.
- text: The text to colour.
+ Keyword arguments:
+ colour: Text colour -- one of the colour constants defined in this
+ class.
+ text: The text to colour.
- Returns:
- If self._enabled is False, returns the original text. If it's True,
- returns text with colour escape sequences based on the value of colour.
- """
- if not self._enabled:
- return text
- if colour == self.BOLD:
- start = self.BOLD_START
- else:
- base = self.BRIGHT_START if bright else self.NORMAL_START
- start = base % (colour + 30)
- return start + text + self.RESET
+ Returns:
+ If self._enabled is False, returns the original text. If it's True,
+ returns text with colour escape sequences based on the value of
+ colour.
+ """
+ if not self._enabled:
+ return text
+ if colour == self.BOLD:
+ start = self.BOLD_START
+ else:
+ base = self.BRIGHT_START if bright else self.NORMAL_START
+ start = base % (colour + 30)
+ return start + text + self.RESET
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 18/20] patman: Correct unit tests to run correctly
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (16 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 17/20] patman: Fix indentation in terminal.py Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 19/20] patman: Remove the -a option Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL Simon Glass
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
It seems that doctest behaves differently now, and some of the unit tests
do not run. Adjust the tests to work correctly.
./tools/patman/patman --test
<unittest.result.TestResult run=10 errors=0 failures=0>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4: None
Changes in v3:
- Add new patch to build current source tree
Changes in v2: None
tools/patman/gitutil.py | 8 ++++----
tools/patman/patchstream.py | 7 +++++--
tools/patman/terminal.py | 8 ++++++--
tools/patman/test.py | 13 +++++++------
4 files changed, 22 insertions(+), 14 deletions(-)
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 18133f2..77cf73f 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -476,13 +476,13 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0):
...
OSError: Recursive email alias at 'other'
>>> LookupEmail('odd', alias, raise_on_error=False)
- \033[1;31mAlias 'odd' not found\033[0m
+ Alias 'odd' not found
[]
>>> # In this case the loop part will effectively be ignored.
>>> LookupEmail('loop', alias, raise_on_error=False)
- \033[1;31mRecursive email alias at 'other'\033[0m
- \033[1;31mRecursive email alias at 'john'\033[0m
- \033[1;31mRecursive email alias at 'mary'\033[0m
+ Recursive email alias at 'other'
+ Recursive email alias at 'john'
+ Recursive email alias at 'mary'
['j.bloggs at napier.co.nz', 'm.poppins at cloud.net']
"""
if not alias:
diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index 0040468..322374c 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -275,7 +275,7 @@ class PatchStream:
# Suppress duplicate signoffs
elif signoff_match:
- if (self.is_log or
+ if (self.is_log or not self.commit or
self.commit.CheckDuplicateSignoff(signoff_match.group(1))):
out = [line]
@@ -312,7 +312,10 @@ class PatchStream:
out = []
log = self.series.MakeChangeLog(self.commit)
out += self.FormatTags(self.tags)
- out += [line] + self.commit.notes + [''] + log
+ out += [line]
+ if self.commit:
+ out += self.commit.notes
+ out += [''] + log
elif self.found_test:
if not re_allowed_after_test.match(line):
self.lines_after_test += 1
diff --git a/tools/patman/terminal.py b/tools/patman/terminal.py
index be216d3..9510dfd 100644
--- a/tools/patman/terminal.py
+++ b/tools/patman/terminal.py
@@ -30,8 +30,12 @@ class Colour(object):
enabled: True if colour output should be enabled. If False then this
class will not add colour codes at all.
"""
- self._enabled = (coloured == COLOUR_ALWAYS or
- (coloured == COLOUR_IF_TERMINAL and os.isatty(sys.stdout.fileno())))
+ try:
+ self._enabled = (coloured == COLOUR_ALWAYS or
+ (coloured == COLOUR_IF_TERMINAL and
+ os.isatty(sys.stdout.fileno())))
+ except:
+ self._enabled = False
def Start(self, colour, bright=True):
"""Returns a start colour code.
diff --git a/tools/patman/test.py b/tools/patman/test.py
index 8fcfe53..e8f7472 100644
--- a/tools/patman/test.py
+++ b/tools/patman/test.py
@@ -55,6 +55,7 @@ This adds functions to enable/disable clocks and reset to on-chip peripherals.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
+
arch/arm/cpu/armv7/tegra2/Makefile | 2 +-
arch/arm/cpu/armv7/tegra2/ap20.c | 57 ++----
arch/arm/cpu/armv7/tegra2/clock.c | 163 +++++++++++++++++
@@ -200,7 +201,7 @@ index 0000000..2234c87
self.assertEqual(result.errors, 0)
self.assertEqual(result.warnings, 0)
self.assertEqual(result.checks, 0)
- self.assertEqual(result.lines, 67)
+ self.assertEqual(result.lines, 56)
os.remove(inf)
def testNoSignoff(self):
@@ -211,18 +212,18 @@ index 0000000..2234c87
self.assertEqual(result.errors, 1)
self.assertEqual(result.warnings, 0)
self.assertEqual(result.checks, 0)
- self.assertEqual(result.lines, 67)
+ self.assertEqual(result.lines, 56)
os.remove(inf)
def testSpaces(self):
inf = self.SetupData('spaces')
result = checkpatch.CheckPatch(inf)
self.assertEqual(result.ok, False)
- self.assertEqual(len(result.problems), 1)
+ self.assertEqual(len(result.problems), 2)
self.assertEqual(result.errors, 0)
- self.assertEqual(result.warnings, 1)
+ self.assertEqual(result.warnings, 2)
self.assertEqual(result.checks, 0)
- self.assertEqual(result.lines, 67)
+ self.assertEqual(result.lines, 56)
os.remove(inf)
def testIndent(self):
@@ -233,7 +234,7 @@ index 0000000..2234c87
self.assertEqual(result.errors, 0)
self.assertEqual(result.warnings, 0)
self.assertEqual(result.checks, 1)
- self.assertEqual(result.lines, 67)
+ self.assertEqual(result.lines, 56)
os.remove(inf)
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 19/20] patman: Remove the -a option
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (17 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 18/20] patman: Correct unit tests to run correctly Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-09 21:33 ` [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL Simon Glass
19 siblings, 0 replies; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
It seems that this is no longer needed, since checkpatch.pl will catch
whitespace problems in patches. Also the option is not widely used, so
it seems safe to just remove it.
Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2:
- Minor changes to the text
tools/patman/gitutil.py | 88 -------------------------------------------------
tools/patman/patman.py | 7 ----
2 files changed, 95 deletions(-)
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 77cf73f..029ad48 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -213,94 +213,6 @@ def CreatePatches(start, count, series):
else:
return None, files
-def ApplyPatch(verbose, fname):
- """Apply a patch with git am to test it
-
- TODO: Convert these to use command, with stderr option
-
- Args:
- fname: filename of patch file to apply
- """
- col = terminal.Colour()
- cmd = ['git', 'am', fname]
- pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE,
- stderr=subprocess.PIPE)
- stdout, stderr = pipe.communicate()
- re_error = re.compile('^error: patch failed: (.+):(\d+)')
- for line in stderr.splitlines():
- if verbose:
- print line
- match = re_error.match(line)
- if match:
- print checkpatch.GetWarningMsg(col, 'warning', match.group(1),
- int(match.group(2)), 'Patch failed')
- return pipe.returncode == 0, stdout
-
-def ApplyPatches(verbose, args, start_point):
- """Apply the patches with git am to make sure all is well
-
- Args:
- verbose: Print out 'git am' output verbatim
- args: List of patch files to apply
- start_point: Number of commits back from HEAD to start applying.
- Normally this is len(args), but it can be larger if a start
- offset was given.
- """
- error_count = 0
- col = terminal.Colour()
-
- # Figure out our current position
- cmd = ['git', 'name-rev', 'HEAD', '--name-only']
- pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE)
- stdout, stderr = pipe.communicate()
- if pipe.returncode:
- str = 'Could not find current commit name'
- print col.Colour(col.RED, str)
- print stdout
- return False
- old_head = stdout.splitlines()[0]
- if old_head == 'undefined':
- str = "Invalid HEAD '%s'" % stdout.strip()
- print col.Colour(col.RED, str)
- return False
-
- # Checkout the required start point
- cmd = ['git', 'checkout', 'HEAD~%d' % start_point]
- pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE,
- stderr=subprocess.PIPE)
- stdout, stderr = pipe.communicate()
- if pipe.returncode:
- str = 'Could not move to commit before patch series'
- print col.Colour(col.RED, str)
- print stdout, stderr
- return False
-
- # Apply all the patches
- for fname in args:
- ok, stdout = ApplyPatch(verbose, fname)
- if not ok:
- print col.Colour(col.RED, 'git am returned errors for %s: will '
- 'skip this patch' % fname)
- if verbose:
- print stdout
- error_count += 1
- cmd = ['git', 'am', '--skip']
- pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE)
- stdout, stderr = pipe.communicate()
- if pipe.returncode != 0:
- print col.Colour(col.RED, 'Unable to skip patch! Aborting...')
- print stdout
- break
-
- # Return to our previous position
- cmd = ['git', 'checkout', old_head]
- pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
- stdout, stderr = pipe.communicate()
- if pipe.returncode:
- print col.Colour(col.RED, 'Could not move back to head commit')
- print stdout, stderr
- return error_count == 0
-
def BuildEmailList(in_list, tag=None, alias=None, raise_on_error=True):
"""Build a list of email addresses based on an input list.
diff --git a/tools/patman/patman.py b/tools/patman/patman.py
index 8fdedda..8b89765 100755
--- a/tools/patman/patman.py
+++ b/tools/patman/patman.py
@@ -25,9 +25,6 @@ import test
parser = OptionParser()
-parser.add_option('-a', '--no-apply', action='store_false',
- dest='apply_patches', default=True,
- help="Don't test-apply patches with git am")
parser.add_option('-H', '--full-help', action='store_true', dest='full_help',
default=False, help='Display the README file')
parser.add_option('-c', '--count', dest='count', type='int',
@@ -144,10 +141,6 @@ else:
ok = checkpatch.CheckPatches(options.verbose, args)
else:
ok = True
- if options.apply_patches:
- if not gitutil.ApplyPatches(options.verbose, args,
- options.count + options.start):
- ok = False
cc_file = series.MakeCcFile(options.process_tags, cover_fname,
not options.ignore_bad_tags)
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL
2014-08-09 21:32 [U-Boot] [PATCH v6 0/20] Add some missing buildman features and deprecate MAKEALL Simon Glass
` (18 preceding siblings ...)
2014-08-09 21:33 ` [U-Boot] [PATCH v6 19/20] patman: Remove the -a option Simon Glass
@ 2014-08-09 21:33 ` Simon Glass
2014-08-11 18:23 ` York Sun
19 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-09 21:33 UTC (permalink / raw)
To: u-boot
Since buildman now includes most of the features of MAKEALL it is probably
time to talk about deprecating MAKEALL.
Comments welcome.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v6: None
Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2:
- Add new patch to fix existing typos
MAKEALL | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/MAKEALL b/MAKEALL
index 929fe88..dbbf74b 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -60,6 +60,10 @@ usage()
exit ${ret}
}
+echo "** Note: MAKEALL is deprecated - please use buildman instead"
+echo "** See tools/buildman/README for details"
+echo
+
SHORT_OPTS="ha:c:v:s:b:lmMCnr"
LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
--
2.0.0.526.g5318336
^ permalink raw reply related [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL
2014-08-09 21:33 ` [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL Simon Glass
@ 2014-08-11 18:23 ` York Sun
2014-08-12 15:00 ` Simon Glass
0 siblings, 1 reply; 34+ messages in thread
From: York Sun @ 2014-08-11 18:23 UTC (permalink / raw)
To: u-boot
On 08/09/2014 02:33 PM, Simon Glass wrote:
> Since buildman now includes most of the features of MAKEALL it is probably
> time to talk about deprecating MAKEALL.
>
> Comments welcome.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
> Changes in v6: None
> Changes in v5: None
> Changes in v4: None
> Changes in v3: None
> Changes in v2:
> - Add new patch to fix existing typos
>
> MAKEALL | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/MAKEALL b/MAKEALL
> index 929fe88..dbbf74b 100755
> --- a/MAKEALL
> +++ b/MAKEALL
> @@ -60,6 +60,10 @@ usage()
> exit ${ret}
> }
>
> +echo "** Note: MAKEALL is deprecated - please use buildman instead"
> +echo "** See tools/buildman/README for details"
> +echo
> +
> SHORT_OPTS="ha:c:v:s:b:lmMCnr"
> LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
>
>
Simon,
This doesn't stop this patch. Just a suggestion.
It may be helpful to print this message at the end of output of MAKEALL. I check
the summary at the end of run carefully, but not paying too much attention at
the beginning. Beside, if I have a lot of output I may miss the beginning.
Another suggestion for future patch is to use return value for buildman. For
now, I have to check if err files are created during building.
York
^ permalink raw reply [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL
2014-08-11 18:23 ` York Sun
@ 2014-08-12 15:00 ` Simon Glass
2014-08-12 21:05 ` Tom Rini
0 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-12 15:00 UTC (permalink / raw)
To: u-boot
Hi York (and Tom),
On 11 August 2014 12:23, York Sun <yorksun@freescale.com> wrote:
> On 08/09/2014 02:33 PM, Simon Glass wrote:
>> Since buildman now includes most of the features of MAKEALL it is probably
>> time to talk about deprecating MAKEALL.
>>
>> Comments welcome.
>>
>> Signed-off-by: Simon Glass <sjg@chromium.org>
>> ---
>>
>> Changes in v6: None
>> Changes in v5: None
>> Changes in v4: None
>> Changes in v3: None
>> Changes in v2:
>> - Add new patch to fix existing typos
>>
>> MAKEALL | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/MAKEALL b/MAKEALL
>> index 929fe88..dbbf74b 100755
>> --- a/MAKEALL
>> +++ b/MAKEALL
>> @@ -60,6 +60,10 @@ usage()
>> exit ${ret}
>> }
>>
>> +echo "** Note: MAKEALL is deprecated - please use buildman instead"
>> +echo "** See tools/buildman/README for details"
>> +echo
>> +
>> SHORT_OPTS="ha:c:v:s:b:lmMCnr"
>> LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
>>
>>
> Simon,
>
> This doesn't stop this patch. Just a suggestion.
>
> It may be helpful to print this message at the end of output of MAKEALL. I check
> the summary at the end of run carefully, but not paying too much attention at
> the beginning. Beside, if I have a lot of output I may miss the beginning.
>
> Another suggestion for future patch is to use return value for buildman. For
> now, I have to check if err files are created during building.
OK I'll take a look at these.
Tom, what do you think about me sending a pull request for the initial
buildman patches just to get them out of the way? It seems that we
might end up tweaking things a bit more and the original patches
probably won't change:
74aef4b buildman: Allow selection of the number of commits to build
75c313d buildman: Introduce an 'and' operator for board selection
e9f8dc3 buildman: Add a few more toolchain examples to the README
09ec73d buildman: Add a message indicating there are no errors
b6817f5 buildman: Add an option to specify the buildman config file
e752166 buildman: Remove unused non-incremental build method code
07f44ff buildman: Add verbose option to display errors as they happen
1d4e7a2 buildman: Refactor output options
e6eeadc buildman: Sort command line options
90d17a6 buildman: Move BuilderThread code to its own file
285691e buildman: Allow building of current source tree
6ed14ca buildman: Add some notes about moving from MAKEALL
ddb01e7 buildman: Fix a few typos
Regards,
Simon
^ permalink raw reply [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL
2014-08-12 15:00 ` Simon Glass
@ 2014-08-12 21:05 ` Tom Rini
2014-08-12 22:22 ` Simon Glass
0 siblings, 1 reply; 34+ messages in thread
From: Tom Rini @ 2014-08-12 21:05 UTC (permalink / raw)
To: u-boot
On Tue, Aug 12, 2014 at 09:00:05AM -0600, Simon Glass wrote:
> Hi York (and Tom),
>
> On 11 August 2014 12:23, York Sun <yorksun@freescale.com> wrote:
> > On 08/09/2014 02:33 PM, Simon Glass wrote:
> >> Since buildman now includes most of the features of MAKEALL it is probably
> >> time to talk about deprecating MAKEALL.
> >>
> >> Comments welcome.
> >>
> >> Signed-off-by: Simon Glass <sjg@chromium.org>
> >> ---
> >>
> >> Changes in v6: None
> >> Changes in v5: None
> >> Changes in v4: None
> >> Changes in v3: None
> >> Changes in v2:
> >> - Add new patch to fix existing typos
> >>
> >> MAKEALL | 4 ++++
> >> 1 file changed, 4 insertions(+)
> >>
> >> diff --git a/MAKEALL b/MAKEALL
> >> index 929fe88..dbbf74b 100755
> >> --- a/MAKEALL
> >> +++ b/MAKEALL
> >> @@ -60,6 +60,10 @@ usage()
> >> exit ${ret}
> >> }
> >>
> >> +echo "** Note: MAKEALL is deprecated - please use buildman instead"
> >> +echo "** See tools/buildman/README for details"
> >> +echo
> >> +
> >> SHORT_OPTS="ha:c:v:s:b:lmMCnr"
> >> LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
> >>
> >>
> > Simon,
> >
> > This doesn't stop this patch. Just a suggestion.
> >
> > It may be helpful to print this message at the end of output of MAKEALL. I check
> > the summary at the end of run carefully, but not paying too much attention at
> > the beginning. Beside, if I have a lot of output I may miss the beginning.
> >
> > Another suggestion for future patch is to use return value for buildman. For
> > now, I have to check if err files are created during building.
>
> OK I'll take a look at these.
>
> Tom, what do you think about me sending a pull request for the initial
> buildman patches just to get them out of the way? It seems that we
> might end up tweaking things a bit more and the original patches
> probably won't change:
>
> 74aef4b buildman: Allow selection of the number of commits to build
> 75c313d buildman: Introduce an 'and' operator for board selection
> e9f8dc3 buildman: Add a few more toolchain examples to the README
> 09ec73d buildman: Add a message indicating there are no errors
> b6817f5 buildman: Add an option to specify the buildman config file
> e752166 buildman: Remove unused non-incremental build method code
> 07f44ff buildman: Add verbose option to display errors as they happen
> 1d4e7a2 buildman: Refactor output options
> e6eeadc buildman: Sort command line options
> 90d17a6 buildman: Move BuilderThread code to its own file
> 285691e buildman: Allow building of current source tree
> 6ed14ca buildman: Add some notes about moving from MAKEALL
> ddb01e7 buildman: Fix a few typos
I'm fine with taking everything but the final part of the series (20/20,
RFC on removing MAKEALL). It'll make further testing of buildman easier
too :)
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140812/765acab2/attachment.pgp>
^ permalink raw reply [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL
2014-08-12 21:05 ` Tom Rini
@ 2014-08-12 22:22 ` Simon Glass
2014-08-12 22:27 ` Tom Rini
0 siblings, 1 reply; 34+ messages in thread
From: Simon Glass @ 2014-08-12 22:22 UTC (permalink / raw)
To: u-boot
Hi Tom,
On 12 August 2014 15:05, Tom Rini <trini@ti.com> wrote:
> On Tue, Aug 12, 2014 at 09:00:05AM -0600, Simon Glass wrote:
>
>> Hi York (and Tom),
>>
>> On 11 August 2014 12:23, York Sun <yorksun@freescale.com> wrote:
>> > On 08/09/2014 02:33 PM, Simon Glass wrote:
>> >> Since buildman now includes most of the features of MAKEALL it is probably
>> >> time to talk about deprecating MAKEALL.
>> >>
>> >> Comments welcome.
>> >>
>> >> Signed-off-by: Simon Glass <sjg@chromium.org>
>> >> ---
>> >>
>> >> Changes in v6: None
>> >> Changes in v5: None
>> >> Changes in v4: None
>> >> Changes in v3: None
>> >> Changes in v2:
>> >> - Add new patch to fix existing typos
>> >>
>> >> MAKEALL | 4 ++++
>> >> 1 file changed, 4 insertions(+)
>> >>
>> >> diff --git a/MAKEALL b/MAKEALL
>> >> index 929fe88..dbbf74b 100755
>> >> --- a/MAKEALL
>> >> +++ b/MAKEALL
>> >> @@ -60,6 +60,10 @@ usage()
>> >> exit ${ret}
>> >> }
>> >>
>> >> +echo "** Note: MAKEALL is deprecated - please use buildman instead"
>> >> +echo "** See tools/buildman/README for details"
>> >> +echo
>> >> +
>> >> SHORT_OPTS="ha:c:v:s:b:lmMCnr"
>> >> LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
>> >>
>> >>
>> > Simon,
>> >
>> > This doesn't stop this patch. Just a suggestion.
>> >
>> > It may be helpful to print this message at the end of output of MAKEALL. I check
>> > the summary at the end of run carefully, but not paying too much attention at
>> > the beginning. Beside, if I have a lot of output I may miss the beginning.
>> >
>> > Another suggestion for future patch is to use return value for buildman. For
>> > now, I have to check if err files are created during building.
>>
>> OK I'll take a look at these.
>>
>> Tom, what do you think about me sending a pull request for the initial
>> buildman patches just to get them out of the way? It seems that we
>> might end up tweaking things a bit more and the original patches
>> probably won't change:
>>
>> 74aef4b buildman: Allow selection of the number of commits to build
>> 75c313d buildman: Introduce an 'and' operator for board selection
>> e9f8dc3 buildman: Add a few more toolchain examples to the README
>> 09ec73d buildman: Add a message indicating there are no errors
>> b6817f5 buildman: Add an option to specify the buildman config file
>> e752166 buildman: Remove unused non-incremental build method code
>> 07f44ff buildman: Add verbose option to display errors as they happen
>> 1d4e7a2 buildman: Refactor output options
>> e6eeadc buildman: Sort command line options
>> 90d17a6 buildman: Move BuilderThread code to its own file
>> 285691e buildman: Allow building of current source tree
>> 6ed14ca buildman: Add some notes about moving from MAKEALL
>> ddb01e7 buildman: Fix a few typos
>
> I'm fine with taking everything but the final part of the series (20/20,
> RFC on removing MAKEALL). It'll make further testing of buildman easier
> too :)
Are you sure? That includes the 'colour' patch at present...
Regards,
Simon
^ permalink raw reply [flat|nested] 34+ messages in thread* [U-Boot] [PATCH v6 20/20] RFC: Deprecate MAKEALL
2014-08-12 22:22 ` Simon Glass
@ 2014-08-12 22:27 ` Tom Rini
0 siblings, 0 replies; 34+ messages in thread
From: Tom Rini @ 2014-08-12 22:27 UTC (permalink / raw)
To: u-boot
On Tue, Aug 12, 2014 at 04:22:36PM -0600, Simon Glass wrote:
> Hi Tom,
>
> On 12 August 2014 15:05, Tom Rini <trini@ti.com> wrote:
> > On Tue, Aug 12, 2014 at 09:00:05AM -0600, Simon Glass wrote:
> >
> >> Hi York (and Tom),
> >>
> >> On 11 August 2014 12:23, York Sun <yorksun@freescale.com> wrote:
> >> > On 08/09/2014 02:33 PM, Simon Glass wrote:
> >> >> Since buildman now includes most of the features of MAKEALL it is probably
> >> >> time to talk about deprecating MAKEALL.
> >> >>
> >> >> Comments welcome.
> >> >>
> >> >> Signed-off-by: Simon Glass <sjg@chromium.org>
> >> >> ---
> >> >>
> >> >> Changes in v6: None
> >> >> Changes in v5: None
> >> >> Changes in v4: None
> >> >> Changes in v3: None
> >> >> Changes in v2:
> >> >> - Add new patch to fix existing typos
> >> >>
> >> >> MAKEALL | 4 ++++
> >> >> 1 file changed, 4 insertions(+)
> >> >>
> >> >> diff --git a/MAKEALL b/MAKEALL
> >> >> index 929fe88..dbbf74b 100755
> >> >> --- a/MAKEALL
> >> >> +++ b/MAKEALL
> >> >> @@ -60,6 +60,10 @@ usage()
> >> >> exit ${ret}
> >> >> }
> >> >>
> >> >> +echo "** Note: MAKEALL is deprecated - please use buildman instead"
> >> >> +echo "** See tools/buildman/README for details"
> >> >> +echo
> >> >> +
> >> >> SHORT_OPTS="ha:c:v:s:b:lmMCnr"
> >> >> LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
> >> >>
> >> >>
> >> > Simon,
> >> >
> >> > This doesn't stop this patch. Just a suggestion.
> >> >
> >> > It may be helpful to print this message at the end of output of MAKEALL. I check
> >> > the summary at the end of run carefully, but not paying too much attention at
> >> > the beginning. Beside, if I have a lot of output I may miss the beginning.
> >> >
> >> > Another suggestion for future patch is to use return value for buildman. For
> >> > now, I have to check if err files are created during building.
> >>
> >> OK I'll take a look at these.
> >>
> >> Tom, what do you think about me sending a pull request for the initial
> >> buildman patches just to get them out of the way? It seems that we
> >> might end up tweaking things a bit more and the original patches
> >> probably won't change:
> >>
> >> 74aef4b buildman: Allow selection of the number of commits to build
> >> 75c313d buildman: Introduce an 'and' operator for board selection
> >> e9f8dc3 buildman: Add a few more toolchain examples to the README
> >> 09ec73d buildman: Add a message indicating there are no errors
> >> b6817f5 buildman: Add an option to specify the buildman config file
> >> e752166 buildman: Remove unused non-incremental build method code
> >> 07f44ff buildman: Add verbose option to display errors as they happen
> >> 1d4e7a2 buildman: Refactor output options
> >> e6eeadc buildman: Sort command line options
> >> 90d17a6 buildman: Move BuilderThread code to its own file
> >> 285691e buildman: Allow building of current source tree
> >> 6ed14ca buildman: Add some notes about moving from MAKEALL
> >> ddb01e7 buildman: Fix a few typos
> >
> > I'm fine with taking everything but the final part of the series (20/20,
> > RFC on removing MAKEALL). It'll make further testing of buildman easier
> > too :)
>
> Are you sure? That includes the 'colour' patch at present...
Ah right, well, not that one since I said we should drop it.
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140812/4b0bb858/attachment.pgp>
^ permalink raw reply [flat|nested] 34+ messages in thread