[PATCH 0/3] recipetool/devtool: send errors to stderr

[PATCH 0/3] recipetool/devtool: send errors to stderr

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

This aligns with the behavior of bitbake proper. There is a bit of code
duplication between knotty and scriptutils, but that's not new -- that was the
case before as well. We should consider moving these logger configuration
functions into bb.msg.

The following changes since commit 2ad7308ee7166641eff99f3b9fe6794de143f6bc:

  oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread start (2015-09-22 18:13:02 +0100)

are available in the git repository at:

  git://github.com/kergoth/openembedded-core scriptutils-log-improvements
  https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvements

Christopher Larson (3):
  oe.scriptutils: enable color in a more flexible way
  oe.scriptutils: add logger_setup_filters
  recipetool, devtool: set up the logging filters

 scripts/devtool            |  1 +
 scripts/lib/scriptutils.py | 29 +++++++++++++++++++++++------
 scripts/recipetool         |  1 +
 3 files changed, 25 insertions(+), 6 deletions(-)

-- 
2.2.1

[PATCH 1/3] oe.scriptutils: enable color in a more flexible way

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

Rather than recreating handlers and forcing them, iterate over the handlers
and enable color on ones we can handle. This makes it easier to handle color
properly when we introduce the bb.msg default log filters.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 scripts/lib/scriptutils.py | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py
index 3366882..3c165eb 100644
--- a/scripts/lib/scriptutils.py
+++ b/scripts/lib/scriptutils.py
@@ -30,12 +30,12 @@ def logger_create(name):
 
 def logger_setup_color(logger, color='auto'):
     from bb.msg import BBLogFormatter
-    console = logging.StreamHandler(sys.stdout)
-    formatter = BBLogFormatter("%(levelname)s: %(message)s")
-    console.setFormatter(formatter)
-    logger.handlers = [console]
-    if color == 'always' or (color=='auto' and console.stream.isatty()):
-        formatter.enable_color()
+
+    for handler in logger.handlers:
+        if (isinstance(handler, logging.StreamHandler) and
+            isinstance(handler.formatter, BBLogFormatter)):
+            if color == 'always' or (color == 'auto' and handler.stream.isatty()):
+                handler.formatter.enable_color()
 
 
 def load_plugins(logger, plugins, pluginpath):
@@ -59,6 +59,7 @@ def load_plugins(logger, plugins, pluginpath):
                 plugin.plugin_init(plugins)
             plugins.append(plugin)
 
+
 def git_convert_standalone_clone(repodir):
     """If specified directory is a git repository, ensure it's a standalone clone"""
     import bb.process
@@ -70,6 +71,7 @@ def git_convert_standalone_clone(repodir):
             bb.process.run('git repack -a', cwd=repodir)
             os.remove(alternatesfile)
 
+
 def fetch_uri(d, uri, destdir, srcrev=None):
     """Fetch a URI to a local directory"""
     import bb.data
-- 
2.2.1

Re: [PATCH 1/3] oe.scriptutils: enable color in a more flexible way

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

On Wed, Sep 23, 2015 at 11:24 AM, Christopher Larson <kergoth@gmail.com>
wrote:

> From: Christopher Larson <chris_larson@mentor.com>
>
> Rather than recreating handlers and forcing them, iterate over the handlers
> and enable color on ones we can handle. This makes it easier to handle
> color
> properly when we introduce the bb.msg default log filters.
>
> Signed-off-by: Christopher Larson <chris_larson@mentor.com>
>

Hmm, might be worth making this even more generic and moving it to bb.msg.
If one was to run this against a logger which isn't configured to use
BBLogFormatter, this would do nothing. It should probably raise an
exception if that occurs, or automatically replace the stock formatter
instance with a BBLogFormatter instance using the same format string.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1473 bytes --]

[PATCH 2/3] oe.scriptutils: add logger_setup_filters

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

This function sets up the default bb.msg formatter and log filters, which by
default sends ERROR messages to stderr rather than stdout.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 scripts/lib/scriptutils.py | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py
index 3c165eb..db0394a 100644
--- a/scripts/lib/scriptutils.py
+++ b/scripts/lib/scriptutils.py
@@ -20,6 +20,7 @@ import os
 import logging
 import glob
 
+
 def logger_create(name):
     logger = logging.getLogger(name)
     loggerhandler = logging.StreamHandler()
@@ -28,6 +29,20 @@ def logger_create(name):
     logger.setLevel(logging.INFO)
     return logger
 
+
+def logger_setup_filters(logger):
+    import bb.msg
+
+    console = logging.StreamHandler(sys.stdout)
+    errconsole = logging.StreamHandler(sys.stderr)
+    bb.msg.addDefaultlogFilter(console, bb.msg.BBLogFilterStdOut)
+    bb.msg.addDefaultlogFilter(errconsole, bb.msg.BBLogFilterStdErr)
+    format_str = "%(levelname)s: %(message)s"
+    console.setFormatter(bb.msg.BBLogFormatter(format_str))
+    errconsole.setFormatter(bb.msg.BBLogFormatter(format_str))
+    logger.handlers = [console, errconsole]
+
+
 def logger_setup_color(logger, color='auto'):
     from bb.msg import BBLogFormatter
 
-- 
2.2.1

[PATCH 3/3] recipetool, devtool: set up the logging filters

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

We want errors going to stderr, the way they do in bitbake itself.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 scripts/devtool    | 1 +
 scripts/recipetool | 1 +
 2 files changed, 2 insertions(+)

diff --git a/scripts/devtool b/scripts/devtool
index 87df951..5146538 100755
--- a/scripts/devtool
+++ b/scripts/devtool
@@ -241,6 +241,7 @@ def main():
         logger.debug('Using standard bitbake path %s' % bitbakepath)
         scriptpath.add_oe_lib_path()
 
+    scriptutils.logger_setup_filters(logger)
     scriptutils.logger_setup_color(logger, global_args.color)
 
     if global_args.bbpath is None:
diff --git a/scripts/recipetool b/scripts/recipetool
index 87fb35e..a1fc6e6 100755
--- a/scripts/recipetool
+++ b/scripts/recipetool
@@ -72,6 +72,7 @@ def main():
         sys.exit(1)
     logger.debug('Found bitbake path: %s' % bitbakepath)
 
+    scriptutils.logger_setup_filters(logger)
     scriptutils.logger_setup_color(logger, global_args.color)
 
     tinfoil = tinfoil_init(False)
-- 
2.2.1

Re: [PATCH 0/3] recipetool/devtool: send errors to stderr

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

On Wed, Sep 23, 2015 at 11:21 AM Christopher Larson <kergoth@gmail.com>
wrote:

> From: Christopher Larson <chris_larson@mentor.com>
>
> This aligns with the behavior of bitbake proper. There is a bit of code
> duplication between knotty and scriptutils, but that's not new -- that was
> the
> case before as well. We should consider moving these logger configuration
> functions into bb.msg.
>
> The following changes since commit
> 2ad7308ee7166641eff99f3b9fe6794de143f6bc:
>
>   oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread
> start (2015-09-22 18:13:02 +0100)
>
> are available in the git repository at:
>
>   git://github.com/kergoth/openembedded-core scriptutils-log-improvements
>
> https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvements
>
> Christopher Larson (3):
>   oe.scriptutils: enable color in a more flexible way
>   oe.scriptutils: add logger_setup_filters
>   recipetool, devtool: set up the logging filters
>

Does anyone happen to know if there any objection to this? I just noticed
this hasn't been merged yet. This makes the behavior of recipetool and
devtool more consistent with bitbake itself by sending errors to stderr.
Long term I'd like to see warnings go there as well, but that could be more
controversial and is a later task, I think this should go in as is.
Eventually we should consolidate some of the logger setup and configuration
logic between scriptutils and bitbake as well, but that was the case with
or without this series

[-- Attachment #2: Type: text/html, Size: 2141 bytes --]

[PATCH 0/3] recipetool/devtool: send errors to stderr

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

On Wed, Sep 23, 2015 at 11:21 AM Christopher Larson <kergoth@gmail.com>
wrote:

> From: Christopher Larson <chris_larson@mentor.com>
>
> This aligns with the behavior of bitbake proper. There is a bit of code
> duplication between knotty and scriptutils, but that's not new -- that was
> the
> case before as well. We should consider moving these logger configuration
> functions into bb.msg.
>
> The following changes since commit
> 2ad7308ee7166641eff99f3b9fe6794de143f6bc:
>
>   oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread
> start (2015-09-22 18:13:02 +0100)
>
> are available in the git repository at:
>
>   git://github.com/kergoth/openembedded-core scriptutils-log-improvements
>
> https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvements
>
> Christopher Larson (3):
>   oe.scriptutils: enable color in a more flexible way
>   oe.scriptutils: add logger_setup_filters
>   recipetool, devtool: set up the logging filters
>

Does anyone happen to know if there any objection to this? I just noticed
this hasn't been merged yet. This makes the behavior of recipetool and
devtool more consistent with bitbake itself by sending errors to stderr.
Long term I'd like to see warnings go there as well, but that could be more
controversial and is a later task, I think this should go in as is.
Eventually we should consolidate some of the logger setup and configuration
logic between scriptutils and bitbake as well, but that was the case with
or without this series

[-- Attachment #2: Type: text/html, Size: 2257 bytes --]

Re: [PATCH 0/3] recipetool/devtool: send errors to stderr

[Paul Eggleton]

On Fri, 11 Dec 2015 17:09:33 Christopher Larson wrote:
> On Wed, Sep 23, 2015 at 11:21 AM Christopher Larson <kergoth@gmail.com>
> 
> wrote:
> > From: Christopher Larson <chris_larson@mentor.com>
> > 
> > This aligns with the behavior of bitbake proper. There is a bit of code
> > duplication between knotty and scriptutils, but that's not new -- that was
> > the
> > case before as well. We should consider moving these logger configuration
> > functions into bb.msg.
> > 
> > The following changes since commit
> > 
> > 2ad7308ee7166641eff99f3b9fe6794de143f6bc:
> >   oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread
> > 
> > start (2015-09-22 18:13:02 +0100)
> > 
> > are available in the git repository at:
> >   git://github.com/kergoth/openembedded-core scriptutils-log-improvements
> > 
> > https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvem
> > ents> 
> > Christopher Larson (3):
> >   oe.scriptutils: enable color in a more flexible way
> >   oe.scriptutils: add logger_setup_filters
> >   recipetool, devtool: set up the logging filters
> 
> Does anyone happen to know if there any objection to this? I just noticed
> this hasn't been merged yet. This makes the behavior of recipetool and
> devtool more consistent with bitbake itself by sending errors to stderr.
> Long term I'd like to see warnings go there as well, but that could be more
> controversial and is a later task, I think this should go in as is.
> Eventually we should consolidate some of the logger setup and configuration
> logic between scriptutils and bitbake as well, but that was the case with
> or without this series

I certainly don't - I did see this hadn't been merged a few weeks back, I 
should have acked it.

Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>

Cheers,
Paul

-- 
Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH 0/3] recipetool/devtool: send errors to stderr

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]

Hi Chris,

On 23 September 2015 at 19:24, Christopher Larson <kergoth@gmail.com> wrote:

> are available in the git repository at:
>
>   git://github.com/kergoth/openembedded-core scriptutils-log-improvements
>
> https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvements
>

There's also a similar scriptutils-log-improvements-2 branch, what branch
is the one you want merged?

Ross

[-- Attachment #2: Type: text/html, Size: 1082 bytes --]

Re: [PATCH 0/3] recipetool/devtool: send errors to stderr

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Mon, Dec 14, 2015 at 5:06 AM, Burton, Ross <ross.burton@intel.com> wrote:

> On 23 September 2015 at 19:24, Christopher Larson <kergoth@gmail.com>
> wrote:
>
>> are available in the git repository at:
>>
>>   git://github.com/kergoth/openembedded-core scriptutils-log-improvements
>>
>> https://github.com/kergoth/openembedded-core/tree/scriptutils-log-improvements
>>
>
> There's also a similar scriptutils-log-improvements-2 branch, what branch
> is the one you want merged?


The scriptutils-log-improvements branch, please. scriptutils-log-improvements-2
includes bits which aren't yet ready. Thanks.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1752 bytes --]