* [PATCH 1/5] [dora] bitbake: cooker/command: Add error state for the server and use for pre_serve errors
2013-12-10 15:09 [PATCH 0/5] [dora] bitbake: fixes for dora Robert Yang
@ 2013-12-10 15:09 ` Robert Yang
2013-12-10 15:09 ` [PATCH 2/5] [dora] bitbake: monitordisk: lower inode check warning to note Robert Yang
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Robert Yang @ 2013-12-10 15:09 UTC (permalink / raw)
To: bitbake-devel
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if errors occur when starting the PR service, there is a race that
occurs since the UI runs various commands including starting builds before
processing the CookerExit(). By adding the error state and refusing to run
async commands in this mode, builds are prevented from starting and the
UI reaches the exit code with the system shutting down cleanly.
(Bitbake master rev: 42fa34142ea685f91115a551e74416ca28ef1c91)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
bitbake/lib/bb/command.py | 2 ++
bitbake/lib/bb/cooker.py | 3 ++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/bitbake/lib/bb/command.py b/bitbake/lib/bb/command.py
index 3ca27a6..a2795ce 100644
--- a/bitbake/lib/bb/command.py
+++ b/bitbake/lib/bb/command.py
@@ -86,6 +86,8 @@ class Command:
def runAsyncCommand(self):
try:
+ if self.cooker.state == bb.cooker.state.error:
+ return False
if self.currentAsyncCommand is not None:
(command, options) = self.currentAsyncCommand
commandmethod = getattr(CommandsAsync, command)
diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index ad36b34..b504f45 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -61,7 +61,7 @@ class CollectionError(bb.BBHandledException):
"""
class state:
- initial, parsing, running, shutdown, forceshutdown, stopped = range(6)
+ initial, parsing, running, shutdown, forceshutdown, stopped, error = range(7)
class SkippedPackage:
@@ -1321,6 +1321,7 @@ class BBCooker:
self.prhost = prserv.serv.auto_start(self.data)
except prserv.serv.PRServiceConfigError:
bb.event.fire(CookerExit(), self.event_data)
+ self.state = state.error
return
def post_serve(self):
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/5] [dora] bitbake: monitordisk: lower inode check warning to note
2013-12-10 15:09 [PATCH 0/5] [dora] bitbake: fixes for dora Robert Yang
2013-12-10 15:09 ` [PATCH 1/5] [dora] bitbake: cooker/command: Add error state for the server and use for pre_serve errors Robert Yang
@ 2013-12-10 15:09 ` Robert Yang
2013-12-10 15:09 ` [PATCH 3/5] [dora] bitbake: fetch2: Fix handling of SCM mirrors in MIRRORS Robert Yang
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Robert Yang @ 2013-12-10 15:09 UTC (permalink / raw)
To: bitbake-devel
From: Olof Johansson <olof.johansson@axis.com>
Filesystems like btrfs and reiserfs sets the inode count to 0, since
they don't have an inode concept. This is expected, and having a warning
show up every time you run bitbake can cause undue concern.
(Bitbake master rev: f3ac2d3678f48c68a250a0a20c08cf8687322d38)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
bitbake/lib/bb/monitordisk.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bitbake/lib/bb/monitordisk.py b/bitbake/lib/bb/monitordisk.py
index 3e6ecbd..fca43ee 100644
--- a/bitbake/lib/bb/monitordisk.py
+++ b/bitbake/lib/bb/monitordisk.py
@@ -243,7 +243,7 @@ class diskMonitor:
# zero, this is a feature of the fs, we disable the inode
# checking for such a fs.
if st.f_files == 0:
- logger.warn("Inode check for %s is unavaliable, will remove it from disk monitor" % path)
+ logger.info("Inode check for %s is unavaliable, will remove it from disk monitor" % path)
self.devDict[k][2] = None
continue
# Always show warning, the self.checked would always be False if the action is WARN
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/5] [dora] bitbake: fetch2: Fix handling of SCM mirrors in MIRRORS
2013-12-10 15:09 [PATCH 0/5] [dora] bitbake: fixes for dora Robert Yang
2013-12-10 15:09 ` [PATCH 1/5] [dora] bitbake: cooker/command: Add error state for the server and use for pre_serve errors Robert Yang
2013-12-10 15:09 ` [PATCH 2/5] [dora] bitbake: monitordisk: lower inode check warning to note Robert Yang
@ 2013-12-10 15:09 ` Robert Yang
2013-12-10 15:09 ` [PATCH 4/5] [dora] bitbake: build.py: add single-quotes around already-expanded directory name Robert Yang
2013-12-10 15:09 ` [PATCH 5/5] [dora] bitbake: runqueue: Fix hole in setsceneverify skipped task logic Robert Yang
4 siblings, 0 replies; 6+ messages in thread
From: Robert Yang @ 2013-12-10 15:09 UTC (permalink / raw)
To: bitbake-devel
From: Richard Purdie <richard.purdie@linuxfoundation.org>
If an SCM mirror is in PREMIRRORS, the tarball is downloaded and then found
by the "upstream" check and handled correctly.
If an SCM mirror is in MIRRORS, the tarball is downloaded but not used
since there is no "upstream" run after MIRRORS completes. It therefore
sits there useless and unused. This code change forces the upstream to
run after a mirror tarball is found and fixes the usage of SCM mirrors
in MIRRORS.
(Bitbake master rev: a66ee0994645aa5658b2f5ea134ed17d89f8751a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
bitbake/lib/bb/fetch2/__init__.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 451d104..b9f673c 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -805,6 +805,10 @@ def try_mirror_url(newuri, origud, ud, ld, check = False):
dest = os.path.join(dldir, os.path.basename(ud.localpath))
if not os.path.exists(dest):
os.symlink(ud.localpath, dest)
+ if not os.path.exists(origud.donestamp) or origud.method.need_update(origud.url, origud, ld):
+ origud.method.download(origud.url, origud, ld)
+ if hasattr(ud.method,"build_mirror_data"):
+ origud.method.build_mirror_data(origud.url, origud, ld)
return None
# Otherwise the result is a local file:// and we symlink to it
if not os.path.exists(origud.localpath):
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 4/5] [dora] bitbake: build.py: add single-quotes around already-expanded directory name
2013-12-10 15:09 [PATCH 0/5] [dora] bitbake: fixes for dora Robert Yang
` (2 preceding siblings ...)
2013-12-10 15:09 ` [PATCH 3/5] [dora] bitbake: fetch2: Fix handling of SCM mirrors in MIRRORS Robert Yang
@ 2013-12-10 15:09 ` Robert Yang
2013-12-10 15:09 ` [PATCH 5/5] [dora] bitbake: runqueue: Fix hole in setsceneverify skipped task logic Robert Yang
4 siblings, 0 replies; 6+ messages in thread
From: Robert Yang @ 2013-12-10 15:09 UTC (permalink / raw)
To: bitbake-devel
From: Peter Seebach <peter.seebach@windriver.com>
If the computed name of a directory contains an undefined variable
reference, bitbake dutifully creates a directory with a name that has
${...} in it. However, the actual task script created then tries to cd
to that directory, and the cd command fails, because no such directory
exists -- because the shell has helpfully removed the ${...} which did
not match any actual variables.
Since we want the name to be used exactly-as-is, add single quotes around
the name so this doesn't cause strange failures running tasks, which
allows us to progress past such failures and get to a point where they
can be diagnosed.
(Bitbake master rev: 2809c2e6f2f35f9b08058950be896947ab5a0284)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
bitbake/lib/bb/build.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
index 2e49a09..edab556 100644
--- a/bitbake/lib/bb/build.py
+++ b/bitbake/lib/bb/build.py
@@ -285,7 +285,7 @@ set -e
if bb.msg.loggerVerboseLogs:
script.write("set -x\n")
if cwd:
- script.write("cd %s\n" % cwd)
+ script.write("cd '%s'\n" % cwd)
script.write("%s\n" % func)
script.write('''
# cleanup
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 5/5] [dora] bitbake: runqueue: Fix hole in setsceneverify skipped task logic
2013-12-10 15:09 [PATCH 0/5] [dora] bitbake: fixes for dora Robert Yang
` (3 preceding siblings ...)
2013-12-10 15:09 ` [PATCH 4/5] [dora] bitbake: build.py: add single-quotes around already-expanded directory name Robert Yang
@ 2013-12-10 15:09 ` Robert Yang
4 siblings, 0 replies; 6+ messages in thread
From: Robert Yang @ 2013-12-10 15:09 UTC (permalink / raw)
To: bitbake-devel
From: Richard Purdie <richard.purdie@linuxfoundation.org>
We have do_bundle_initramfs which is a task inserted after compile and
before build. It is not covered by sstate.
If we run a build with a valid sstate cache present, the setsceneverify
function realises it will rerun the do_compile step (due to the
bundle_initramfs task) and hence marks do_populate_sysroot to rerun.
do_install, a dependency of do_populate_sysroot is left as marked as
covered by sstate.
What we need to do is traverse the dependency tree for any setsceneverify
invalided task and ensure any dependencies are also invalidated. We can
stop at any point we reach another setscene task though.
This means the do_populate_sysroot task has the data from do_install
available and doesn't crash.
(Bitbake master rev: f21910157d873c030b149c4cdc5b57c5062ab5a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
bitbake/lib/bb/runqueue.py | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index c09cfd4..72c0208 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1204,6 +1204,8 @@ class RunQueueExecuteTasks(RunQueueExecute):
self.stampcache = {}
+ initial_covered = self.rq.scenequeue_covered.copy()
+
# Mark initial buildable tasks
for task in xrange(self.stats.total):
self.runq_running.append(0)
@@ -1257,13 +1259,28 @@ class RunQueueExecuteTasks(RunQueueExecute):
except TypeError:
covered_remove = bb.utils.better_eval(call2, locs)
- for task in covered_remove:
+ def removecoveredtask(task):
fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]]
taskname = self.rqdata.runq_task[task] + '_setscene'
bb.build.del_stamp(taskname, self.rqdata.dataCache, fn)
- logger.debug(1, 'Not skipping task %s due to setsceneverify', task)
self.rq.scenequeue_covered.remove(task)
+ toremove = covered_remove
+ for task in toremove:
+ logger.debug(1, 'Not skipping task %s due to setsceneverify', task)
+ while toremove:
+ covered_remove = []
+ for task in toremove:
+ removecoveredtask(task)
+ for deptask in self.rqdata.runq_depends[task]:
+ if deptask not in self.rq.scenequeue_covered:
+ continue
+ if deptask in toremove or deptask in covered_remove or deptask in initial_covered:
+ continue
+ logger.debug(1, 'Task %s depends on task %s so not skipping' % (task, deptask))
+ covered_remove.append(deptask)
+ toremove = covered_remove
+
logger.debug(1, 'Full skip list %s', self.rq.scenequeue_covered)
event.fire(bb.event.StampUpdate(self.rqdata.target_pairs, self.rqdata.dataCache.stamp), self.cfgData)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread