[PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

Ensures that recipetool and devtool pull plugins from layers, and
oe-selftest will pull tests from layers.

Unrelated to that, also prevents a traceback on parsing failure, and adds
a tiny feature to appendsrc to facilitate reuse by other sub-commands.

[v3 Update] Fixed the commit message for `devtool: also load plugins from
BBPATH` to match the implementation. Split up `recipetool: also load plugins
from BBPATH` into two commits with clear commit messages.

[v2 Update] `devtool: also load plugins from BBPATH` was fixed, so it no longer
breaks the devtool selftests.

The following changes since commit 3143920c541b55b543b9dcc12b18af4e0e4b7ae1:

  linux-libc-headers: update to 4.1 (2015-07-23 08:47:52 +0100)

are available in the git repository at:

  git@github.com:kergoth/openembedded-core yocto-bug-7625

for you to fetch changes up to 6ac90ef048adff65b6dce1e4102156ccb365964a:

  oe-selftest: add libdirs from BBPATH to sys.path (2015-07-23 07:59:18 -0700)

----------------------------------------------------------------
Christopher Larson (7):
      recipetool: catch BBHandledException from parsing
      recipetool.append: add extralines arg to appendsrc
      recipetool: also load plugins from BBPATH
      recipetool: parse global args early
      devtool: also load plugins from BBPATH
      oe-selftest: obey oeqa.selftest.__path__
      oe-selftest: add libdirs from BBPATH to sys.path

 scripts/devtool                  | 57 +++++++++++++++++++++++++---------------
 scripts/lib/devtool/__init__.py  |  4 +--
 scripts/lib/recipetool/append.py |  4 +--
 scripts/oe-selftest              | 18 +++++++++----
 scripts/recipetool               | 57 +++++++++++++++++++++++++---------------
 5 files changed, 89 insertions(+), 51 deletions(-)

-- 
2.2.1

[PATCHv3 1/7] recipetool: catch BBHandledException from parsing

[Christopher Larson]

This ensures that we don't see a traceback on parsing failures.

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 scripts/recipetool | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/scripts/recipetool b/scripts/recipetool
index c68bef4..3063cf7 100755
--- a/scripts/recipetool
+++ b/scripts/recipetool
@@ -82,9 +82,11 @@ def main():
 
     scriptutils.logger_setup_color(logger, args.color)
 
-    tinfoil_init(getattr(args, 'parserecipes', False))
-
-    ret = args.func(args)
+    try:
+        tinfoil_init(getattr(args, 'parserecipes', False))
+        ret = args.func(args)
+    except bb.BBHandledException:
+        ret = 1
 
     return ret
 
-- 
2.2.1

[PATCHv3 2/7] recipetool.append: add extralines arg to appendsrc

[Christopher Larson]

This makes the function more reusable for other sub-commands.

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 scripts/lib/recipetool/append.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/recipetool/append.py b/scripts/lib/recipetool/append.py
index 3f2f9a4..ed7d0d4 100644
--- a/scripts/lib/recipetool/append.py
+++ b/scripts/lib/recipetool/append.py
@@ -337,7 +337,7 @@ def appendfile(args):
         return 3
 
 
-def appendsrc(args, files, rd):
+def appendsrc(args, files, rd, extralines=None):
     import oe.recipeutils
 
     srcdir = rd.getVar('S', True)
@@ -352,7 +352,7 @@ def appendsrc(args, files, rd):
         simplified[str(simple_uri)] = uri
 
     copyfiles = {}
-    extralines = []
+    extralines = extralines or []
     for newfile, srcfile in files.iteritems():
         src_destdir = os.path.dirname(srcfile)
         if not args.use_workdir:
-- 
2.2.1

[PATCHv3 3/7] recipetool: also load plugins from BBPATH

[Christopher Larson]

This makes it easier to extend, as a layer can add its own sub-commands.

The bitbake path setup is moved earlier, as it has to be done before
tinfoil_init.

[YOCTO #7625]

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 scripts/recipetool | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/scripts/recipetool b/scripts/recipetool
index 3063cf7..37c2dd3 100755
--- a/scripts/recipetool
+++ b/scripts/recipetool
@@ -36,11 +36,8 @@ def tinfoil_init(parserecipes):
     import logging
     tinfoil = bb.tinfoil.Tinfoil()
     tinfoil.prepare(not parserecipes)
-
-    for plugin in plugins:
-        if hasattr(plugin, 'tinfoil_init'):
-            plugin.tinfoil_init(tinfoil)
     tinfoil.logger.setLevel(logger.getEffectiveLevel())
+    return tinfoil
 
 def main():
 
@@ -55,12 +52,26 @@ def main():
     parser.add_argument('--color', choices=['auto', 'always', 'never'], default='auto', help='Colorize output (where %(metavar)s is %(choices)s)', metavar='COLOR')
     subparsers = parser.add_subparsers(title='subcommands', metavar='<subcommand>')
 
-    scriptutils.load_plugins(logger, plugins, os.path.join(scripts_path, 'lib', 'recipetool'))
+    import scriptpath
+    bitbakepath = scriptpath.add_bitbake_lib_path()
+    if not bitbakepath:
+        logger.error("Unable to find bitbake by searching parent directory of this script or PATH")
+        sys.exit(1)
+    logger.debug('Found bitbake path: %s' % bitbakepath)
+
+    tinfoil = tinfoil_init(False)
+    for path in ([scripts_path] +
+                 tinfoil.config_data.getVar('BBPATH', True).split(':')):
+        pluginpath = os.path.join(path, 'lib', 'recipetool')
+        scriptutils.load_plugins(logger, plugins, pluginpath)
+
     registered = False
     for plugin in plugins:
         if hasattr(plugin, 'register_command'):
             registered = True
             plugin.register_command(subparsers)
+        if hasattr(plugin, 'tinfoil_init'):
+            plugin.tinfoil_init(tinfoil)
 
     if not registered:
         logger.error("No commands registered - missing plugins?")
@@ -73,17 +84,11 @@ def main():
     elif args.quiet:
         logger.setLevel(logging.ERROR)
 
-    import scriptpath
-    bitbakepath = scriptpath.add_bitbake_lib_path()
-    if not bitbakepath:
-        logger.error("Unable to find bitbake by searching parent directory of this script or PATH")
-        sys.exit(1)
-    logger.debug('Found bitbake path: %s' % bitbakepath)
-
     scriptutils.logger_setup_color(logger, args.color)
 
     try:
-        tinfoil_init(getattr(args, 'parserecipes', False))
+        if getattr(args, 'parserecipes', False):
+            tinfoil.parseRecipes()
         ret = args.func(args)
     except bb.BBHandledException:
         ret = 1
-- 
2.2.1

[PATCHv3 4/7] recipetool: parse global args early

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

This separates the argument parsing into two steps, which lets us apply global
settings like enabling debugging before the plugins load, so we can see the
paths where plugins are being loaded.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 scripts/recipetool | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/scripts/recipetool b/scripts/recipetool
index 37c2dd3..87fb35e 100755
--- a/scripts/recipetool
+++ b/scripts/recipetool
@@ -46,12 +46,25 @@ def main():
         sys.exit(1)
 
     parser = argparse.ArgumentParser(description="OpenEmbedded recipe tool",
+                                     add_help=False,
                                      epilog="Use %(prog)s <subcommand> --help to get help on a specific command")
     parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true')
     parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true')
     parser.add_argument('--color', choices=['auto', 'always', 'never'], default='auto', help='Colorize output (where %(metavar)s is %(choices)s)', metavar='COLOR')
+
+    global_args, unparsed_args = parser.parse_known_args()
+
+    # Help is added here rather than via add_help=True, as we don't want it to
+    # be handled by parse_known_args()
+    parser.add_argument('-h', '--help', action='help', default=argparse.SUPPRESS,
+                        help='show this help message and exit')
     subparsers = parser.add_subparsers(title='subcommands', metavar='<subcommand>')
 
+    if global_args.debug:
+        logger.setLevel(logging.DEBUG)
+    elif global_args.quiet:
+        logger.setLevel(logging.ERROR)
+
     import scriptpath
     bitbakepath = scriptpath.add_bitbake_lib_path()
     if not bitbakepath:
@@ -59,6 +72,8 @@ def main():
         sys.exit(1)
     logger.debug('Found bitbake path: %s' % bitbakepath)
 
+    scriptutils.logger_setup_color(logger, global_args.color)
+
     tinfoil = tinfoil_init(False)
     for path in ([scripts_path] +
                  tinfoil.config_data.getVar('BBPATH', True).split(':')):
@@ -77,14 +92,7 @@ def main():
         logger.error("No commands registered - missing plugins?")
         sys.exit(1)
 
-    args = parser.parse_args()
-
-    if args.debug:
-        logger.setLevel(logging.DEBUG)
-    elif args.quiet:
-        logger.setLevel(logging.ERROR)
-
-    scriptutils.logger_setup_color(logger, args.color)
+    args = parser.parse_args(unparsed_args, namespace=global_args)
 
     try:
         if getattr(args, 'parserecipes', False):
-- 
2.2.1

[PATCHv3 5/7] devtool: also load plugins from BBPATH

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

This makes it easier to extend, as a layer can add its own sub-commands.

Argument parsing is also separated into two steps, the same way it's done in
recipetool, as we need access to the global command-line arguments early,
before plugins are loaded, both for debugging arguments and for the bitbake
path (we need to load the bitbake module to get tinfoil, which is now needed
to load the plugins).

Rather than constructing tinfoil once and passing it through into sub-commands
for their use, we have to construct it for configuration metadata, use it, and
then shut it down, as some sub-commands call out to recipetool, which needs
its own tinfoil instance, and therefore needs to acquire the bitbake lock. If
we're still holding the lock at that point, that's clearly a problem.

[YOCTO #7625]

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 scripts/devtool                 | 57 ++++++++++++++++++++++++++---------------
 scripts/lib/devtool/__init__.py |  4 +--
 2 files changed, 38 insertions(+), 23 deletions(-)

diff --git a/scripts/devtool b/scripts/devtool
index fa799f6..557a830 100755
--- a/scripts/devtool
+++ b/scripts/devtool
@@ -35,7 +35,7 @@ context = None
 scripts_path = os.path.dirname(os.path.realpath(__file__))
 lib_path = scripts_path + '/lib'
 sys.path = sys.path + [lib_path]
-from devtool import DevtoolError
+from devtool import DevtoolError, setup_tinfoil
 import scriptutils
 logger = scriptutils.logger_create('devtool')
 
@@ -186,37 +186,28 @@ def main():
         pth = os.path.dirname(pth)
 
     parser = argparse.ArgumentParser(description="OpenEmbedded development tool",
+                                     add_help=False,
                                      epilog="Use %(prog)s <subcommand> --help to get help on a specific command")
     parser.add_argument('--basepath', help='Base directory of SDK / build directory')
     parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true')
     parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true')
     parser.add_argument('--color', choices=['auto', 'always', 'never'], default='auto', help='Colorize output (where %(metavar)s is %(choices)s)', metavar='COLOR')
 
-    subparsers = parser.add_subparsers(dest="subparser_name", title='subcommands', metavar='<subcommand>')
+    global_args, unparsed_args = parser.parse_known_args()
 
-    if not context.fixed_setup:
-        parser_create_workspace = subparsers.add_parser('create-workspace',
-                                                        help='Set up a workspace',
-                                                        description='Sets up a new workspace. NOTE: other devtool subcommands will create a workspace automatically as needed, so you only need to use %(prog)s if you want to specify where the workspace should be located.')
-        parser_create_workspace.add_argument('layerpath', nargs='?', help='Path in which the workspace layer should be created')
-        parser_create_workspace.add_argument('--create-only', action="store_true", help='Only create the workspace layer, do not alter configuration')
-        parser_create_workspace.set_defaults(func=create_workspace)
+    # Help is added here rather than via add_help=True, as we don't want it to
+    # be handled by parse_known_args()
+    parser.add_argument('-h', '--help', action='help', default=argparse.SUPPRESS,
+                        help='show this help message and exit')
 
-    scriptutils.load_plugins(logger, plugins, os.path.join(scripts_path, 'lib', 'devtool'))
-    for plugin in plugins:
-        if hasattr(plugin, 'register_commands'):
-            plugin.register_commands(subparsers, context)
-
-    args = parser.parse_args()
-
-    if args.debug:
+    if global_args.debug:
         logger.setLevel(logging.DEBUG)
-    elif args.quiet:
+    elif global_args.quiet:
         logger.setLevel(logging.ERROR)
 
-    if args.basepath:
+    if global_args.basepath:
         # Override
-        basepath = args.basepath
+        basepath = global_args.basepath
     elif not context.fixed_setup:
         basepath = os.environ.get('BUILDDIR')
         if not basepath:
@@ -246,7 +237,31 @@ def main():
         logger.debug('Using standard bitbake path %s' % bitbakepath)
         scriptpath.add_oe_lib_path()
 
-    scriptutils.logger_setup_color(logger, args.color)
+    scriptutils.logger_setup_color(logger, global_args.color)
+
+    tinfoil = setup_tinfoil(config_only=True)
+    for path in ([scripts_path] +
+                 tinfoil.config_data.getVar('BBPATH', True).split(':')):
+        pluginpath = os.path.join(path, 'lib', 'devtool')
+        scriptutils.load_plugins(logger, plugins, pluginpath)
+    tinfoil.cooker.shutdown(force=True)
+    tinfoil.cooker.unlockBitbake()
+
+    subparsers = parser.add_subparsers(dest="subparser_name", title='subcommands', metavar='<subcommand>')
+
+    if not context.fixed_setup:
+        parser_create_workspace = subparsers.add_parser('create-workspace',
+                                                        help='Set up a workspace',
+                                                        description='Sets up a new workspace. NOTE: other devtool subcommands will create a workspace automatically as needed, so you only need to use %(prog)s if you want to specify where the workspace should be located.')
+        parser_create_workspace.add_argument('layerpath', nargs='?', help='Path in which the workspace layer should be created')
+        parser_create_workspace.add_argument('--create-only', action="store_true", help='Only create the workspace layer, do not alter configuration')
+        parser_create_workspace.set_defaults(func=create_workspace)
+
+    for plugin in plugins:
+        if hasattr(plugin, 'register_commands'):
+            plugin.register_commands(subparsers, context)
+
+    args = parser.parse_args(unparsed_args, namespace=global_args)
 
     if args.subparser_name != 'create-workspace':
         read_workspace()
diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 61b810c..b54ddf5 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -96,7 +96,7 @@ def exec_fakeroot(d, cmd, **kwargs):
             newenv[splitval[0]] = splitval[1]
     return subprocess.call("%s %s" % (fakerootcmd, cmd), env=newenv, **kwargs)
 
-def setup_tinfoil():
+def setup_tinfoil(config_only=False):
     """Initialize tinfoil api from bitbake"""
     import scriptpath
     bitbakepath = scriptpath.add_bitbake_lib_path()
@@ -106,7 +106,7 @@ def setup_tinfoil():
 
     import bb.tinfoil
     tinfoil = bb.tinfoil.Tinfoil()
-    tinfoil.prepare(False)
+    tinfoil.prepare(config_only)
     tinfoil.logger.setLevel(logger.getEffectiveLevel())
     return tinfoil
 
-- 
2.2.1

[PATCHv3 6/7] oe-selftest: obey oeqa.selftest.__path__

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

This ensures that all paths that hold selftest tests will be checked
(oeqa.selftest is a namespace package).

[YOCTO #7625]

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 scripts/oe-selftest | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index fa6245a..c19c692 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -142,11 +142,12 @@ def get_tests(exclusive_modules=[], include_hidden=False):
     for x in exclusive_modules:
         testslist.append('oeqa.selftest.' + x)
     if not testslist:
-        testpath = os.path.abspath(os.path.dirname(oeqa.selftest.__file__))
-        files = sorted([f for f in os.listdir(testpath) if f.endswith('.py') and not (f.startswith('_') and not include_hidden) and not f.startswith('__') and f != 'base.py'])
-        for f in files:
-            module = 'oeqa.selftest.' + f[:-3]
-            testslist.append(module)
+        for testpath in oeqa.selftest.__path__:
+            files = sorted([f for f in os.listdir(testpath) if f.endswith('.py') and not (f.startswith('_') and not include_hidden) and not f.startswith('__') and f != 'base.py'])
+            for f in files:
+                module = 'oeqa.selftest.' + f[:-3]
+                if module not in testslist:
+                    testslist.append(module)
 
     return testslist
 
-- 
2.2.1

[PATCHv3 7/7] oe-selftest: add libdirs from BBPATH to sys.path

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

This ensures that oeqa.selftest.* from layers are found.

[YOCTO #7625]

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 scripts/oe-selftest | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index c19c692..60f9bb8 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -155,6 +155,13 @@ def main():
     parser = get_args_parser()
     args = parser.parse_args()
 
+    # Add <layer>/lib to sys.path, so layers can add selftests
+    log.info("Running bitbake -e to get BBPATH")
+    bbpath = get_bb_var('BBPATH').split(':')
+    layer_libdirs = [p for p in (os.path.join(l, 'lib') for l in bbpath) if os.path.exists(p)]
+    sys.path.extend(layer_libdirs)
+    reload(oeqa.selftest)
+
     if args.list_allclasses:
         args.list_modules = True
 
-- 
2.2.1

Re: [PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[Paul Eggleton]

On Thursday 23 July 2015 12:57:55 Christopher Larson wrote:
> From: Christopher Larson <chris_larson@mentor.com>
> 
> Ensures that recipetool and devtool pull plugins from layers, and
> oe-selftest will pull tests from layers.
> 
> Unrelated to that, also prevents a traceback on parsing failure, and adds
> a tiny feature to appendsrc to facilitate reuse by other sub-commands.
> 
> [v3 Update] Fixed the commit message for `devtool: also load plugins from
> BBPATH` to match the implementation. Split up `recipetool: also load plugins
> from BBPATH` into two commits with clear commit messages.

Looks good, thanks!

Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[Richard Purdie]

On Thu, 2015-07-23 at 12:57 -0700, Christopher Larson wrote:
> From: Christopher Larson <chris_larson@mentor.com>
> 
> Ensures that recipetool and devtool pull plugins from layers, and
> oe-selftest will pull tests from layers.
> 
> Unrelated to that, also prevents a traceback on parsing failure, and adds
> a tiny feature to appendsrc to facilitate reuse by other sub-commands.
> 
> [v3 Update] Fixed the commit message for `devtool: also load plugins from
> BBPATH` to match the implementation. Split up `recipetool: also load plugins
> from BBPATH` into two commits with clear commit messages.
> 
> [v2 Update] `devtool: also load plugins from BBPATH` was fixed, so it no longer
> breaks the devtool selftests.

I know there are some issues with oe-selftest on the autobuilder at the
moment, specifically master results in things like:

https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/93

however when I apply this and your other series, the autobuilder
oe-selftest does this:

https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/92/steps/Running%20oe-selftest/logs/stdio

which is substantially worse. I can't immediately see what the issue is
but the two runs above pretty much bisect this down to the patch series.
I'm therefore reluctant to merge these until we can figure out what is
going on...

Cheers,

Richard

Re: [PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1945 bytes --]

On Sat, Jul 25, 2015 at 9:09 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2015-07-23 at 12:57 -0700, Christopher Larson wrote:
> > From: Christopher Larson <chris_larson@mentor.com>
> >
> > Ensures that recipetool and devtool pull plugins from layers, and
> > oe-selftest will pull tests from layers.
> >
> > Unrelated to that, also prevents a traceback on parsing failure, and adds
> > a tiny feature to appendsrc to facilitate reuse by other sub-commands.
> >
> > [v3 Update] Fixed the commit message for `devtool: also load plugins from
> > BBPATH` to match the implementation. Split up `recipetool: also load
> plugins
> > from BBPATH` into two commits with clear commit messages.
> >
> > [v2 Update] `devtool: also load plugins from BBPATH` was fixed, so it no
> longer
> > breaks the devtool selftests.
>
> I know there are some issues with oe-selftest on the autobuilder at the
> moment, specifically master results in things like:
>
>
> https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/93
>
> however when I apply this and your other series, the autobuilder
> oe-selftest does this:
>
>
> https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/92/steps/Running%20oe-selftest/logs/stdio
>
> which is substantially worse. I can't immediately see what the issue is
> but the two runs above pretty much bisect this down to the patch series.
> I'm therefore reluctant to merge these until we can figure out what is
> going on...


Given that both failures have nothing to do with devtool, recipetool, or
oe-selftest and are simply bitbake build failures as called by tests, I
don’t really see how these could have anything to do with that, but
understood.
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2778 bytes --]

Re: [PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[Richard Purdie]

On Sat, 2015-07-25 at 12:18 -0700, Christopher Larson wrote:
> 
> On Sat, Jul 25, 2015 at 9:09 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>         however when I apply this and your other series, the
>         autobuilder
>         oe-selftest does this:
>         
>         https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/92/steps/Running%20oe-selftest/logs/stdio
>         
>         which is substantially worse. I can't immediately see what the
>         issue is
>         but the two runs above pretty much bisect this down to the
>         patch series.
>         I'm therefore reluctant to merge these until we can figure out
>         what is
>         going on...
> 
> Given that both failures have nothing to do with devtool, recipetool,
> or oe-selftest and are simply bitbake build failures as called by
> tests, I don’t really see how these could have anything to do with
> that, but understood.

FWIW, the issue is that you use the "xxx:do_unpack" syntax in one of the
tests. This breaks toaster.bbclass and once its broken, it stays broken
as its doing a split(":") on data in a file it never cleans up if the
function fails.

I've taken a fix which should avoid these issues. Will rerun a build
with that applied (and various other fixes) and hopefully we can then
get things merged.

Cheers,

Richard

Re: [PATCHv3 0/6] recipetool/devtool/oe-selftest: pull from BBPATH

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1817 bytes --]

On Mon, Jul 27, 2015 at 5:16 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Sat, 2015-07-25 at 12:18 -0700, Christopher Larson wrote:
> >
> > On Sat, Jul 25, 2015 at 9:09 AM, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> >         however when I apply this and your other series, the
> >         autobuilder
> >         oe-selftest does this:
> >
> >
> https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/92/steps/Running%20oe-selftest/logs/stdio
> >
> >         which is substantially worse. I can't immediately see what the
> >         issue is
> >         but the two runs above pretty much bisect this down to the
> >         patch series.
> >         I'm therefore reluctant to merge these until we can figure out
> >         what is
> >         going on...
> >
> > Given that both failures have nothing to do with devtool, recipetool,
> > or oe-selftest and are simply bitbake build failures as called by
> > tests, I don’t really see how these could have anything to do with
> > that, but understood.
>
> FWIW, the issue is that you use the "xxx:do_unpack" syntax in one of the
> tests. This breaks toaster.bbclass and once its broken, it stays broken
> as its doing a split(":") on data in a file it never cleans up if the
> function fails.
>
> I've taken a fix which should avoid these issues. Will rerun a build
> with that applied (and various other fixes) and hopefully we can then
> get things merged.
>

Ah! That explains it. Thanks. Obviously I could have used -c, but I’m a fan
of the new syntax, it gives some nice flexibility :)
-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2698 bytes --]