[PATCH] bitbake/lib/bb/fetch2: Fix stderr handling when running commands

[PATCH] bitbake/lib/bb/fetch2: Fix stderr handling when running commands

[Adrien Bustany]

[-- Attachment #1: Type: text/plain, Size: 454 bytes --]


stderr would previously be appended to stdout, corrupting the result
when something was outputed to stderr but exit code was still 0
(non-fatal warning messages). This commit makes the code parse only
stdout, but output stderr if an error happened.

Signed-off-by: Adrien Bustany <adrien.bustany@nokia.com>
---
 bitbake/lib/bb/fetch2/__init__.py |   35 ++++++++++++++++-------------------
 1 files changed, 16 insertions(+), 19 deletions(-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-bitbake-lib-bb-fetch2-Fix-stderr-handling-when-runni.patch --]
[-- Type: text/x-patch; name="0001-bitbake-lib-bb-fetch2-Fix-stderr-handling-when-runni.patch", Size: 2184 bytes --]

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 374ac9b..c3b22d7 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -378,6 +378,9 @@ def runfetchcmd(cmd, d, quiet = False, cleanup = []):
     Optionally remove the files/directories listed in cleanup upon failure
     """
 
+    import bb.process
+    import subprocess
+
     # Need to export PATH as binary could be in metadata paths
     # rather than host provided
     # Also include some other variables.
@@ -394,33 +397,27 @@ def runfetchcmd(cmd, d, quiet = False, cleanup = []):
 
     logger.debug(1, "Running %s", cmd)
 
-    # redirect stderr to stdout
-    stdout_handle = os.popen(cmd + " 2>&1", "r")
-    output = ""
-
-    while True:
-        line = stdout_handle.readline()
-        if not line:
-            break
-        if not quiet:
-            print(line, end=' ')
-        output += line
+    success = False
+    error_message = ""
 
-    status = stdout_handle.close() or 0
-    signal = status >> 8
-    exitstatus = status & 0xff
+    try:
+        (output, errors) = bb.process.run(cmd, shell=True, stderr=subprocess.PIPE)
+        success = True
+    except bb.process.NotFoundError as e:
+        error_message = "Fetch command %s" % (e.command)
+    except bb.process.CmdError as e:
+        error_message = "Fetch command %s could not be run:\n%s" % (e.command, e.msg)
+    except bb.process.ExecutionError as e:
+        error_message = "Fetch command %s failed with exit code %s, output:\n%s" % (e.command, e.exitcode, e.stderr)
 
-    if (signal or status != 0):
+    if not success:
         for f in cleanup:
             try:
                 bb.utils.remove(f, True)
             except OSError:
                 pass
 
-        if signal:
-            raise FetchError("Fetch command %s failed with signal %s, output:\n%s" % (cmd, signal, output))
-        elif status != 0:
-            raise FetchError("Fetch command %s failed with exit code %s, output:\n%s" % (cmd, status, output))
+        raise FetchError(error_message)
 
     return output
 

Re: [PATCH] bitbake/lib/bb/fetch2: Fix stderr handling when running commands

[Richard Purdie]

On Thu, 2012-01-19 at 17:49 +0200, Adrien Bustany wrote:
> stderr would previously be appended to stdout, corrupting the result
> when something was outputed to stderr but exit code was still 0
> (non-fatal warning messages). This commit makes the code parse only
> stdout, but output stderr if an error happened.
> 
> Signed-off-by: Adrien Bustany <adrien.bustany@nokia.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py |   35 ++++++++++++++++-------------------
>  1 files changed, 16 insertions(+), 19 deletions(-)

Just for reference, I tried to apply this but couldn't as it conflicts
with other changes in that area.

If you have time to rebase it, I'll likely apply it sooner, else I'll
need to find time to do that.

Cheers,

Richard