[PATCH V3 0/1] bitbake: runqueue: catch HashValidateFailure errors in RunQueueExecuteScenequeue

[PATCH V3 0/1] bitbake: runqueue: catch HashValidateFailure errors in RunQueueExecuteScenequeue

[Hongxu Jia]

Changed in V3:
- Add exception HashValidateFailure for the specific error

Changed in V2: 
- Strip the commit message within 80 characters a line

//Hongxu

The following changes since commit 52b788c6df9201764130ab744bf67b770b24b896:

  autogen-native: inherit pkgconfig to fix a build failure (2014-08-25 10:26:00 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib hongxu/bitbake
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/bitbake

Hongxu Jia (1):
  bitbake: runqueue: catch HashValidateFailure errors in
    RunQueueExecuteScenequeue

 bitbake/lib/bb/runqueue.py | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

-- 
1.9.1

[PATCH 1/1] bitbake: runqueue: catch HashValidateFailure errors in RunQueueExecuteScenequeue

[Hongxu Jia]

We need to catch the failure of self.rq.hashvalidate which invoked
in RunQueueExecuteScenequeue, and exit in a normal build.

So we add exception HashValidateFailure for this specific error.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 bitbake/lib/bb/runqueue.py | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index e13dc57..b48b02b 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1046,7 +1046,15 @@ class RunQueue:
                 self.state = runQueueComplete
             else:
                 self.start_worker()
-                self.rqexe = RunQueueExecuteScenequeue(self)
+                try:
+                    self.rqexe = RunQueueExecuteScenequeue(self)
+                except bb.runqueue.HashValidateFailure:
+                    # While HashValidateFailure, we need exit the normal build.
+                    logger.error("Hash validatation failed in RunQueueExecuteScenequeue")
+                    self.state = runQueueComplete
+                except:
+                    logger.error("RunQueueExecuteScenequeue init failed")
+                    raise
 
         if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
             self.dm.check(self)
@@ -1821,7 +1829,10 @@ class RunQueueExecuteScenequeue(RunQueueExecute):
                 sq_task.append(task)
             call = self.rq.hashvalidate + "(sq_fn, sq_task, sq_hash, sq_hashfn, d)"
             locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname, "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
-            valid = bb.utils.better_eval(call, locs)
+            try:
+                valid = bb.utils.better_eval(call, locs)
+            except:
+                raise bb.runqueue.HashValidateFailure()
 
             valid_new = stamppresent
             for v in valid:
@@ -2003,6 +2014,8 @@ class TaskFailure(Exception):
     def __init__(self, x):
         self.args = x
 
+# Exception raised for a hash validation error
+class HashValidateFailure(Exception): pass
 
 class runQueueExitWait(bb.event.Event):
     """
-- 
1.9.1

Re: [PATCH 1/1] bitbake: runqueue: catch HashValidateFailure errors in RunQueueExecuteScenequeue

[Richard Purdie]

On Wed, 2014-08-27 at 19:54 +0800, Hongxu Jia wrote:
> We need to catch the failure of self.rq.hashvalidate which invoked
> in RunQueueExecuteScenequeue, and exit in a normal build.
> 
> So we add exception HashValidateFailure for this specific error.
> 
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  bitbake/lib/bb/runqueue.py | 17 +++++++++++++++--
>  1 file changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> index e13dc57..b48b02b 100644
> --- a/bitbake/lib/bb/runqueue.py
> +++ b/bitbake/lib/bb/runqueue.py
> @@ -1046,7 +1046,15 @@ class RunQueue:
>                  self.state = runQueueComplete
>              else:
>                  self.start_worker()
> -                self.rqexe = RunQueueExecuteScenequeue(self)
> +                try:
> +                    self.rqexe = RunQueueExecuteScenequeue(self)
> +                except bb.runqueue.HashValidateFailure:
> +                    # While HashValidateFailure, we need exit the normal build.
> +                    logger.error("Hash validatation failed in RunQueueExecuteScenequeue")
> +                    self.state = runQueueComplete
> +                except:
> +                    logger.error("RunQueueExecuteScenequeue init failed")
> +                    raise
>  
>          if self.state in [runQueueSceneRun, runQueueRunning, runQueueCleanUp]:
>              self.dm.check(self)
> @@ -1821,7 +1829,10 @@ class RunQueueExecuteScenequeue(RunQueueExecute):
>                  sq_task.append(task)
>              call = self.rq.hashvalidate + "(sq_fn, sq_task, sq_hash, sq_hashfn, d)"
>              locs = { "sq_fn" : sq_fn, "sq_task" : sq_taskname, "sq_hash" : sq_hash, "sq_hashfn" : sq_hashfn, "d" : self.cooker.data }
> -            valid = bb.utils.better_eval(call, locs)
> +            try:
> +                valid = bb.utils.better_eval(call, locs)
> +            except:
> +                raise bb.runqueue.HashValidateFailure()

How about:

except Exception as e:
	logger.error("Hash validation failed in RunQueueExecuteScenequeue %s" % str(e))
        raise BBHandledException

Raising the HandledException error should shut down the system
gracefully from there. Adding a custom exception here doesn't help much
other than warning about the exact point of failure as we can do that in
the logger.error message.

I think if you made this:

except Exception as e:
	logger.critial("Hash validation failed in RunQueueExecuteScenequeue %s" % str(e))

then we'd not even need to BBHandledException as logger.critical does
that internally but I could be wrong, I'm going from memory.

Cheers,

Richard


>              valid_new = stamppresent
>              for v in valid:
> @@ -2003,6 +2014,8 @@ class TaskFailure(Exception):
>      def __init__(self, x):
>          self.args = x
>  
> +# Exception raised for a hash validation error
> +class HashValidateFailure(Exception): pass
>  
>  class runQueueExitWait(bb.event.Event):
>      """