All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Aníbal Limón" <anibal.limon@linux.intel.com>
To: yocto@yoctoproject.org
Cc: joshua.g.lock@intel.com
Subject: [[yocto-autobuilder][PATCHv2] 02/15] lib/buildsteps.py: Add BitbakeLogLineObserver for BitbakeShellCommands.
Date: Tue, 21 Jun 2016 18:07:39 -0500	[thread overview]
Message-ID: <1466550472-1365-3-git-send-email-anibal.limon@linux.intel.com> (raw)
In-Reply-To: <1466550472-1365-1-git-send-email-anibal.limon@linux.intel.com>

The BitbakeLogLineObserver look into stdio output of bitbake process and
identify for exception/errors into log,

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 .../site-packages/autobuilder/lib/buildsteps.py    | 51 ++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/lib/python2.7/site-packages/autobuilder/lib/buildsteps.py b/lib/python2.7/site-packages/autobuilder/lib/buildsteps.py
index 4f15dcf..9822b3c 100644
--- a/lib/python2.7/site-packages/autobuilder/lib/buildsteps.py
+++ b/lib/python2.7/site-packages/autobuilder/lib/buildsteps.py
@@ -11,8 +11,10 @@ __email__ = "anibal.limon@linux.intel.com"
 '''
 
 import os
+import re
 
 from buildbot.steps.shell import ShellCommand
+from buildbot.process.buildstep import LogLineObserver
 
 DEFAULT_SHELL = 'bash'
 
@@ -42,6 +44,55 @@ class ShellCommandCleanEnv(ShellCommand):
 
         return "env -i %s %s -c " % (pe_cmd, shell)
 
+class BitbakeLogLineObserver(LogLineObserver):
+    """
+        Search in the stdio bitbake log for exception/errors, identify
+        if the error is a recipe/task one if not turn on the flag of
+        bitbake error/exception and save the log.
+    """
+
+    rexp_error = re.compile("^ERROR: .*$")
+
+    # recipe task errors regex'es
+    rexp_pnpv_error = re.compile("^ERROR: (?P<pnpv>.*) do_(?P<task>.*):.*$")
+    rexp_task_error = re.compile("^ERROR: Task .*$")
+    rexp_log_error = re.compile("^ERROR: Logfile of failure stored in: "\
+            "(?P<path>.*)$")
+
+    def _handleError(self, line):
+        if not hasattr(self.step, 'errors'):
+            self.step.errors = {}
+            self.step.errors['bitbake'] = False
+            self.step.errors['log'] = []
+
+        # save all the log for be able to get report variables like
+        # machine, target, distro, etc
+        self.step.errors['log'].append(line)
+
+        # discard line that are not errors and line that
+        # is recipe task errors
+        if (not self.rexp_error.match(line)) or \
+                self.rexp_pnpv_error.match(line) or \
+                self.rexp_task_error.match(line) or \
+                self.rexp_log_error.match(line):
+            return
+
+        # if not match recipe task type is a bitbake exception/error
+        self.step.errors['bitbake'] = True
+
+    def outLineReceived(self, line):
+        self._handleError(line)
+
+    def errLineReceived(self, line):
+        self._handleError(line)
+
 class BitbakeShellCommand(ShellCommand):
     def __init__(self, factory, argdict=None, **kwargs):
         super(BitbakeShellCommand, self).__init__(**kwargs)
+
+        self.stdio_observer = BitbakeLogLineObserver()
+        self.addLogObserver('stdio', self.stdio_observer)
+
+    def commandComplete(self, cmd):
+        if cmd.didFail():
+            pass
-- 
2.1.4



  parent reply	other threads:[~2016-06-21 23:08 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-06-21 23:07 [[yocto-autobuilder][PATCHv2] 00/15] Add support for generate bitbake/oe-selftest error reports Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 01/15] lib/buildsteps.py: Add BitbakeShellCommand class Aníbal Limón
2016-06-21 23:07 ` Aníbal Limón [this message]
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 03/15] lib/buildsteps: BitbakeShellCommand add support for create error reports Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 04/15] lib/ABTools: Add functions for get errordir path in controller and worker Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 05/15] buildbot: Add support for DirectoryDownload transfer step Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 06/15] autobuilder/buildsteps: Add DownloadBitbakeErrorReports step Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 07/15] buiidsets: Add DownloadBitbakeErrorReport step Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 08/15] lib/buildsteps.py: BitbakeShellCommand add support for error_type Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 09/15] autobuilder/lib/buildsteps.py: BitbakeShellCommand update error report types Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 10/15] autobuilder/lib/{ABtools, builsteps}.py: Create save_error_report func Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 11/15] autobuilder/lib/ABTools.py: Add get_lsb_distro function Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 12/15] buildsteps/RunOeSelftest.py: Add support for create/save error reports Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 13/15] nightly-oe-selftest.conf: Add steps for Download and Send error report Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 14/15] autobuilder/buildsteps: Rename DownloadBitbakeErrorReports to DownloadErrorReports Aníbal Limón
2016-06-21 23:07 ` [[yocto-autobuilder][PATCHv2] 15/15] buildset-config: Update references from " Aníbal Limón
2016-06-22  7:44 ` [[yocto-autobuilder][PATCHv2] 00/15] Add support for generate bitbake/oe-selftest error reports Beth 'pidge' Flanagan
2016-06-22 20:17   ` Aníbal Limón
2016-06-23 15:27     ` Aníbal Limón
2016-06-23 15:35       ` Beth 'pidge' Flanagan
2016-06-27 12:28 ` Beth 'pidge' Flanagan
2016-06-28 14:55   ` Aníbal Limón

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1466550472-1365-3-git-send-email-anibal.limon@linux.intel.com \
    --to=anibal.limon@linux.intel.com \
    --cc=joshua.g.lock@intel.com \
    --cc=yocto@yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.