[PATCH 0/5] misc Toaster fixes

[PATCH 0/5] misc Toaster fixes

[Ed Bartosh]

Hi,

This patchset contains bug fix for bug 10185 - Image builds not showing as complete in the UI after finishing
and bunch of small related and unrelated fixes for toaster script and toasterui.

Please, review.

The following changes since commit 8dfcfbeb14a179a5d5722007d7c986da1dd41b40:

  arch-mips: Add mips64-o32 tunes (2016-09-24 07:30:10 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/toaster/misc-fixes-toasterui
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/misc-fixes-toasterui

Ed Bartosh (5):
  toaster: stop modifying OEROOT in toaster script
  toaster: check if file exist
  toaster: fix handling of EnvironmentError
  toaster: fix 'Unhandled MetadataEvent' error
  toaster: make error message more informative

 bitbake/bin/toaster                  | 10 ++++++----
 bitbake/lib/bb/ui/buildinfohelper.py |  3 +++
 bitbake/lib/bb/ui/toasterui.py       | 15 +++++++++++----
 3 files changed, 20 insertions(+), 8 deletions(-)

--
Regards,
ed

[PATCH 1/5] toaster: stop modifying OEROOT in toaster script

[Ed Bartosh]

Setting OEROOT in toaster script makes oe-init-build-env to
break with error:
  bash: ../bitbake/bin/../../scripts/oe-buildenv-internal: No such file or directory
This happens because OEROOT contains path relative to build
directory.

Renamed OEROOT to OE_ROOT and unset it after it's used.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/bin/toaster | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/bitbake/bin/toaster b/bitbake/bin/toaster
index 6641dbc..f92d38e 100755
--- a/bitbake/bin/toaster
+++ b/bitbake/bin/toaster
@@ -129,7 +129,7 @@ fi
 
 export BBBASEDIR=`dirname $TOASTER`/..
 MANAGE="python3 $BBBASEDIR/lib/toaster/manage.py"
-OEROOT=`dirname $TOASTER`/../..
+OE_ROOT=`dirname $TOASTER`/../..
 
 # this is the configuraton file we are using for toaster
 # we are using the same logic that oe-setup-builddir uses
@@ -139,16 +139,18 @@ OEROOT=`dirname $TOASTER`/../..
 # in the local layers that currently make using an arbitrary
 # toasterconf.json difficult.
 
-. $OEROOT/.templateconf
+. $OE_ROOT/.templateconf
 if [ -n "$TEMPLATECONF" ]; then
     if [ ! -d "$TEMPLATECONF" ]; then
         # Allow TEMPLATECONF=meta-xyz/conf as a shortcut
-        if [ -d "$OEROOT/$TEMPLATECONF" ]; then
-            TEMPLATECONF="$OEROOT/$TEMPLATECONF"
+        if [ -d "$OE_ROOT/$TEMPLATECONF" ]; then
+            TEMPLATECONF="$OE_ROOT/$TEMPLATECONF"
         fi
     fi
 fi
 
+unset OE_ROOT
+
 # this defines the dir toaster will use for
 # 1) clones of layers (in _toaster_clones )
 # 2) the build dir (in build)
-- 
2.1.4

[PATCH 2/5] toaster: check if file exist

[Ed Bartosh]

Buildinfohelper assumes that all files mentioned in
manifest exist in deploy/ directory, which is not always
the case. Toaster crashes with OSError trying to
call os.stat on non-existing file.

Checking if file exists before processing it should
fix this.

[YOCTO #10185]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/bb/ui/buildinfohelper.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index b2c74dd..970a941 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -1758,6 +1758,9 @@ class BuildInfoHelper(object):
 
                 for basename in basenames:
                     artifact_path = os.path.join(deploy_dir_image, basename)
+                    if not os.path.exists(artifact_path):
+                        logger.warning("artifact %s doesn't exist, skipping" % artifact_path)
+                        continue
                     artifact_size = os.stat(artifact_path).st_size
 
                     # note that the artifact will only be saved against this
-- 
2.1.4

[PATCH 3/5] toaster: fix handling of EnvironmentError

[Ed Bartosh]

Due to the bug in processing EnvironmentError exception,
toasterui ignores it. As EnvironmentError is a base for OSError
and IOError this means that all OSError and IOError exceptions
were silently ignored.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/bb/ui/toasterui.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/ui/toasterui.py b/bitbake/lib/bb/ui/toasterui.py
index b5422cf..569957a 100644
--- a/bitbake/lib/bb/ui/toasterui.py
+++ b/bitbake/lib/bb/ui/toasterui.py
@@ -450,9 +450,12 @@ def main(server, eventHandler, params):
             return_value += 1
 
         except EnvironmentError as ioerror:
-            # ignore interrupted io
-            if ioerror.args[0] == 4:
-                pass
+            logger.warning("EnvironmentError: %s" % ioerror)
+            # ignore interrupted io system calls
+            if ioerror.args[0] == 4: # errno 4 is EINTR
+                logger.warning("Skipped EINTR: %s" % ioerror)
+            else:
+                raise
         except KeyboardInterrupt:
             if params.observe_only:
                 print("\nKeyboard Interrupt, exiting observer...")
-- 
2.1.4

[PATCH 4/5] toaster: fix 'Unhandled MetadataEvent' error

[Ed Bartosh]

New MetadataEvent 'TaskArtifacts' causes this error.
Processing of this event will hopefully be implemented in future.
For now it should be enough to just skip it.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/bb/ui/toasterui.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/bitbake/lib/bb/ui/toasterui.py b/bitbake/lib/bb/ui/toasterui.py
index 569957a..2bc45b6 100644
--- a/bitbake/lib/bb/ui/toasterui.py
+++ b/bitbake/lib/bb/ui/toasterui.py
@@ -431,6 +431,10 @@ def main(server, eventHandler, params):
                     buildinfohelper.scan_sdk_artifacts(event)
                 elif event.type == "SetBRBE":
                     buildinfohelper.brbe = buildinfohelper._get_data_from_event(event)
+                elif event.type == "TaskArtifacts":
+                    # not implemented yet
+                    # see https://bugzilla.yoctoproject.org/show_bug.cgi?id=10283 for details
+                    pass
                 elif event.type == "OSErrorException":
                     logger.error(event)
                 else:
-- 
2.1.4

[PATCH 5/5] toaster: make error message more informative

[Ed Bartosh]

Error message
 ERROR: Unprocessed MetadataEvent <bb.event.MetadataEvent object at 0x7f750e671a58>
doesn't give a lot of information about the event. It just prints
event object, which is always bb.event.MetadataEvent.

Including event type into the error message should make it more
informative:
 ERROR: Unprocessed MetadataEvent TaskArtifacts

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/bb/ui/toasterui.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/ui/toasterui.py b/bitbake/lib/bb/ui/toasterui.py
index 2bc45b6..9808f6b 100644
--- a/bitbake/lib/bb/ui/toasterui.py
+++ b/bitbake/lib/bb/ui/toasterui.py
@@ -438,7 +438,7 @@ def main(server, eventHandler, params):
                 elif event.type == "OSErrorException":
                     logger.error(event)
                 else:
-                    logger.error("Unprocessed MetadataEvent %s ", str(event))
+                    logger.error("Unprocessed MetadataEvent %s", event.type)
                 continue
 
             if isinstance(event, bb.cooker.CookerExit):
-- 
2.1.4

Re: [PATCH 5/5] toaster: make error message more informative

[Michael Wood]

Thanks for these patches, they've been sent upstream and applied to 
toaster-next.

Michael

On 24/09/16 15:50, Ed Bartosh wrote:
> Error message
>   ERROR: Unprocessed MetadataEvent <bb.event.MetadataEvent object at 0x7f750e671a58>
> doesn't give a lot of information about the event. It just prints
> event object, which is always bb.event.MetadataEvent.
>
> Including event type into the error message should make it more
> informative:
>   ERROR: Unprocessed MetadataEvent TaskArtifacts
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>   bitbake/lib/bb/ui/toasterui.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/ui/toasterui.py b/bitbake/lib/bb/ui/toasterui.py
> index 2bc45b6..9808f6b 100644
> --- a/bitbake/lib/bb/ui/toasterui.py
> +++ b/bitbake/lib/bb/ui/toasterui.py
> @@ -438,7 +438,7 @@ def main(server, eventHandler, params):
>                   elif event.type == "OSErrorException":
>                       logger.error(event)
>                   else:
> -                    logger.error("Unprocessed MetadataEvent %s ", str(event))
> +                    logger.error("Unprocessed MetadataEvent %s", event.type)
>                   continue
>   
>               if isinstance(event, bb.cooker.CookerExit):