[PATCH 1/2] fetch2: make fetch failure errors more readable

[PATCH 1/2] fetch2: make fetch failure errors more readable

[Paul Eggleton]

Most of the time we don't need to see the fetch command; the fetch log
includes the command as a debug message in any case, so omit it. Also
adjust the way command output is printed (we don't need stderr/stdout
labelled, and print "no output" instead of "output:\nNone" when there is
no output.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/lib/bb/fetch2/__init__.py |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 37f7d75..1acb0a4 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -471,7 +471,13 @@ def runfetchcmd(cmd, d, quiet = False, cleanup = []):
     except bb.process.NotFoundError as e:
         error_message = "Fetch command %s" % (e.command)
     except bb.process.ExecutionError as e:
-        error_message = "Fetch command %s failed with exit code %s, output:\nSTDOUT: %s\nSTDERR: %s" % (e.command, e.exitcode, e.stdout, e.stderr)
+        if e.stdout:
+            output = "output:\n%s\n%s" % (e.stdout, e.stderr)
+        elif e.stderr:
+            output = "output:\n%s" % e.stderr
+        else:
+            output = "no output"
+        error_message = "Fetch command failed with exit code %s, %s" % (e.exitcode, output)
     except bb.process.CmdError as e:
         error_message = "Fetch command %s could not be run:\n%s" % (e.command, e.msg)
     if not success:
-- 
1.7.9.5

[PATCH 2/2] fetch2: improve error output for checksum failures

[Paul Eggleton]

* Don't print the full exception in the initial warning - if we later
  succeed in fetching the file from a mirror, we won't usually need the
  details (which are in the fetch log if they are needed); otherwise the
  full error will be printed when the fetch operation fails. Also adjust
  the conditional block so that we don't print another warning just
  mentioning we're going to try mirrors.
* Call logger.error() so that with knotty the full log is not printed
* Provide an explanation around the lines we print for easily updating
  the checksums in the recipe. We don't want users to be just blindly
  updating the recipe in case of a transient failure or deliberately
  altered remote file.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/lib/bb/fetch2/__init__.py |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 1acb0a4..a713418 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -358,7 +358,7 @@ def verify_checksum(u, ud, d):
         mismatch = True;
 
     if mismatch:
-        msg = msg + '\nYour checksums:\nSRC_URI[%s] = "%s"\nSRC_URI[%s] = "%s"' % (ud.md5_name, md5data, ud.sha256_name, sha256data)
+        msg = msg + '\nIf this change is expected (e.g. you have upgraded to a new version without updating the checksums) then you can use these lines within the recipe:\nSRC_URI[%s] = "%s"\nSRC_URI[%s] = "%s"\nOtherwise you should retry the download and/or check with upstream to determine if the file has become corrupted or otherwise unexpectedly modified.\n' % (ud.md5_name, md5data, ud.sha256_name, sha256data)
 
     if len(msg):
         raise ChecksumError('Checksum mismatch!%s' % msg, u)
@@ -1204,8 +1204,9 @@ class Fetch(object):
 
                     except BBFetchException as e:
                         if isinstance(e, ChecksumError):
-                            logger.warn("Checksum error encountered with download (will attempt other sources): %s" % str(e))
-                        if isinstance(e, NoChecksumError):
+                            logger.warn("Checksum failure encountered with download of %s - will attempt other sources if available" % u)
+                            logger.debug(1, str(e))
+                        elif isinstance(e, NoChecksumError):
                             raise
                         else:
                             logger.warn('Failed to fetch URL %s, attempting MIRRORS if available' % u)
@@ -1227,6 +1228,8 @@ class Fetch(object):
             except BBFetchException as e:
                 if isinstance(e, NoChecksumError):
                     logger.error("%s" % str(e))
+                elif isinstance(e, ChecksumError):
+                    logger.error("Checksum failure fetching %s" % u)
                 raise
 
             finally:
-- 
1.7.9.5

Re: [PATCH 1/2] fetch2: make fetch failure errors more readable

[Richard Purdie]

On Sun, 2012-09-23 at 18:05 +0100, Paul Eggleton wrote:
> Most of the time we don't need to see the fetch command; the fetch log
> includes the command as a debug message in any case, so omit it. Also
> adjust the way command output is printed (we don't need stderr/stdout
> labelled, and print "no output" instead of "output:\nNone" when there is
> no output.
> 
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py |    8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)

Merged to master, thanks.

Richard