[PATCH 0/1] Avoid build failures due to setscene errors

[PATCH 0/1] Avoid build failures due to setscene errors

[Peter Kjellerstedt]

Occasionally, we see errors on our autobuilders where a setscene task
fails to retrieve a file from our global sstate cache. It typically
looks something like this:

WARNING: zip-3.0-r2 do_populate_sysroot_setscene: Failed to fetch URL
file://66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz;\
downloadfilename=66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz, attempting
MIRRORS if available
ERROR: zip-3.0-r2 do_populate_sysroot_setscene: Fetcher failure:
Unable to find file
file://66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz;\
downloadfilename=66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz anywhere. The
paths that were searched were:
    /home/pkj/.openembedded/sstate-cache
ERROR: zip-3.0-r2 do_populate_sysroot_setscene: No suitable staging
package found
WARNING: Setscene task
(meta/recipes-extended/zip/zip_3.0.bb:do_populate_sysroot_setscene)
failed with exit code '1' - real task will be run instead

As the last warning indicates, the build will proceed and the real
task will run and the build will eventually complete. However, due to
the two errors above, bitbake will return with an error code which
causes the autobuilder to treat the build as failed and it proceeds to
throw everything it built away.

Since this is quite pointless and causes unnecessary build resources
to be spent and grief from the developers, the two patches in this
change set turn the errors from setscene tasks into warnings.

Well, there is actually only one patch in this change set, as the
other patch is for OE-Core...

//Peter

The following changes since commit bc2e0b2e9b95707d96c840dade12b00e1450ecc3:

  libsdl: Move PACKAGECONFIG options from meta-mingw (2017-08-29 12:23:10 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib pkj/setscene-errors-bitbake
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/setscene-errors-bitbake

Peter Kjellerstedt (1):
  fetch2: Allow Fetch.download() to warn instead of error

 bitbake/lib/bb/fetch2/__init__.py | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

-- 
2.12.0

[PATCH 1/1] fetch2: Allow Fetch.download() to warn instead of error

[Peter Kjellerstedt]

Under some situations it can be allowed for Fetch.download() to fail
to fetch a file without causing bitbake to fail. By adding
only_warn=True as argument to Fetch.download(), it will call
logger.warning() instead of logger.error() and thus not cause build
failures.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 bitbake/lib/bb/fetch2/__init__.py | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 3eb0e4d211..58f65ada84 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -1608,9 +1608,10 @@ class Fetch(object):
 
         return local
 
-    def download(self, urls=None):
+    def download(self, urls=None, only_warn=False):
         """
-        Fetch all urls
+        Fetch all urls. In case only_warn is True, a failure to fetch a url
+        will only result in a warning message, rather than an error message.
         """
         if not urls:
             urls = self.urls
@@ -1688,19 +1689,28 @@ class Fetch(object):
 
                 if not localpath or ((not os.path.exists(localpath)) and localpath.find("*") == -1):
                     if firsterr:
-                        logger.error(str(firsterr))
+                        if only_warn:
+                            logger.warning(str(firsterr))
+                        else:
+                            logger.error(str(firsterr))
                     raise FetchError("Unable to fetch URL from any source.", u)
 
                 update_stamp(ud, self.d)
 
             except IOError as e:
                 if e.errno in [os.errno.ESTALE]:
-                    logger.error("Stale Error Observed %s." % u)
+                    if only_warn:
+                        logger.warning("Stale Error Observed %s." % u)
+                    else:
+                        logger.error("Stale Error Observed %s." % u)
                     raise ChecksumError("Stale Error Detected")
 
             except BBFetchException as e:
                 if isinstance(e, ChecksumError):
-                    logger.error("Checksum failure fetching %s" % u)
+                    if only_warn:
+                        logger.warning("Checksum failure fetching %s" % u)
+                    else:
+                        logger.error("Checksum failure fetching %s" % u)
                 raise
 
             finally:
-- 
2.12.0