* [PATCH 0/2] A couple of fixes to error handling @ 2012-02-23 15:42 Paul Eggleton 2012-02-23 15:42 ` [PATCH 1/2] bitbake: fix parse errors not being reported Paul Eggleton 2012-02-23 15:42 ` [PATCH 2/2] bitbake: add file and line number to ParseError Paul Eggleton 0 siblings, 2 replies; 6+ messages in thread From: Paul Eggleton @ 2012-02-23 15:42 UTC (permalink / raw) To: bitbake-devel A couple of patches to fix/improve error handling during parsing. The patches (against poky, but apply cleanly with -p2 against bitbake master) are available at: git://git.yoctoproject.org/poky-contrib paule/bitbake-errors2 http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bitbake-errors2 Paul Eggleton (2): bitbake: fix parse errors not being reported bitbake: add file and line number to ParseError bitbake/lib/bb/cooker.py | 16 ++++++---------- bitbake/lib/bb/parse/__init__.py | 15 +++++++++++++-- bitbake/lib/bb/parse/ast.py | 8 ++++---- bitbake/lib/bb/parse/parse_py/BBHandler.py | 6 ++---- bitbake/lib/bb/parse/parse_py/ConfHandler.py | 11 ++++++----- 5 files changed, 31 insertions(+), 25 deletions(-) -- 1.7.5.4 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/2] bitbake: fix parse errors not being reported 2012-02-23 15:42 [PATCH 0/2] A couple of fixes to error handling Paul Eggleton @ 2012-02-23 15:42 ` Paul Eggleton 2012-02-23 16:14 ` Chris Larson 2012-02-23 15:42 ` [PATCH 2/2] bitbake: add file and line number to ParseError Paul Eggleton 1 sibling, 1 reply; 6+ messages in thread From: Paul Eggleton @ 2012-02-23 15:42 UTC (permalink / raw) To: bitbake-devel Fixes a regression introduced in BitBake rev c9f58ef6b897d3fa5b0d23734b5f2cb3dabb057a which prevents errors during parsing from being fully reported because BitBake shuts down before it can print them. Move the error printing to before the shutdown in order to fix it. Also, fix handling of general unexpected exceptions during parsing (i.e. print out a usable stack trace). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> --- bitbake/lib/bb/cooker.py | 14 +++++--------- 1 files changed, 5 insertions(+), 9 deletions(-) diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index f0778e5..5653874 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -1605,22 +1605,18 @@ class CookerParser(object): self.shutdown() return False except ParsingFailure as exc: - self.shutdown(clean=False) - bb.fatal('Unable to parse %s: %s' % + logger.error('Unable to parse %s: %s' % (exc.recipe, bb.exceptions.to_string(exc.realexception))) + self.shutdown(clean=False) except (bb.parse.ParseError, bb.data_smart.ExpansionError) as exc: + logger.error(str(exc)) self.shutdown(clean=False) - bb.fatal(str(exc)) except SyntaxError as exc: - self.shutdown(clean=False) logger.error('Unable to parse %s', exc.recipe) - sys.exit(1) + self.shutdown(clean=False) except Exception as exc: - etype, value, tb = sys.exc_info() - logger.error('Unable to parse %s', value.recipe, - exc_info=(etype, value, exc.traceback)) + logger.error('Error during parsing %s: %s\n%s' % (exc.recipe, str(exc), ''.join(bb.exceptions.format_extracted(exc.traceback, limit=5)))) self.shutdown(clean=False) - sys.exit(1) self.current += 1 self.virtuals += len(result) -- 1.7.5.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] bitbake: fix parse errors not being reported 2012-02-23 15:42 ` [PATCH 1/2] bitbake: fix parse errors not being reported Paul Eggleton @ 2012-02-23 16:14 ` Chris Larson 2012-02-23 17:23 ` Paul Eggleton 0 siblings, 1 reply; 6+ messages in thread From: Chris Larson @ 2012-02-23 16:14 UTC (permalink / raw) To: Paul Eggleton; +Cc: bitbake-devel On Thu, Feb 23, 2012 at 8:42 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote: > except Exception as exc: > - etype, value, tb = sys.exc_info() > - logger.error('Unable to parse %s', value.recipe, > - exc_info=(etype, value, exc.traceback)) > + logger.error('Error during parsing %s: %s\n%s' % (exc.recipe, str(exc), ''.join(bb.exceptions.format_extracted(exc.traceback, limit=5)))) This seems odd to me. Is the UI not displaying the traceback when sent to it via the exc_info? Half the point of coming up with the new traceback format which is pickleable was to allow us to do exception formatting in the UI. -- Christopher Larson ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] bitbake: fix parse errors not being reported 2012-02-23 16:14 ` Chris Larson @ 2012-02-23 17:23 ` Paul Eggleton 2012-02-23 17:26 ` Chris Larson 0 siblings, 1 reply; 6+ messages in thread From: Paul Eggleton @ 2012-02-23 17:23 UTC (permalink / raw) To: Chris Larson; +Cc: bitbake-devel On Thursday 23 February 2012 09:14:16 Chris Larson wrote: > <paul.eggleton@linux.intel.com> wrote: > > except Exception as exc: > > - etype, value, tb = sys.exc_info() > > - logger.error('Unable to parse %s', value.recipe, > > - exc_info=(etype, value, exc.traceback)) > > + logger.error('Error during parsing %s: %s\n%s' % (exc.recipe, > > str(exc), ''.join(bb.exceptions.format_extracted(exc.traceback, > > limit=5)))) > This seems odd to me. Is the UI not displaying the traceback when sent > to it via the exc_info? Half the point of coming up with the new > traceback format which is pickleable was to allow us to do exception > formatting in the UI. Actually that's odd, I could have sworn when I tried this it broke, but it seems to be doing the right thing now. Will send a v2 reverting this part. Presumably you are happy with the rest of it? Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] bitbake: fix parse errors not being reported 2012-02-23 17:23 ` Paul Eggleton @ 2012-02-23 17:26 ` Chris Larson 0 siblings, 0 replies; 6+ messages in thread From: Chris Larson @ 2012-02-23 17:26 UTC (permalink / raw) To: Paul Eggleton; +Cc: bitbake-devel On Thu, Feb 23, 2012 at 10:23 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote: > On Thursday 23 February 2012 09:14:16 Chris Larson wrote: >> <paul.eggleton@linux.intel.com> wrote: >> > except Exception as exc: >> > - etype, value, tb = sys.exc_info() >> > - logger.error('Unable to parse %s', value.recipe, >> > - exc_info=(etype, value, exc.traceback)) >> > + logger.error('Error during parsing %s: %s\n%s' % (exc.recipe, >> > str(exc), ''.join(bb.exceptions.format_extracted(exc.traceback, >> > limit=5)))) >> This seems odd to me. Is the UI not displaying the traceback when sent >> to it via the exc_info? Half the point of coming up with the new >> traceback format which is pickleable was to allow us to do exception >> formatting in the UI. > > Actually that's odd, I could have sworn when I tried this it broke, but it > seems to be doing the right thing now. Will send a v2 reverting this part. > > Presumably you are happy with the rest of it? Yeah, looks fine to me, nice work. -- Christopher Larson ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] bitbake: add file and line number to ParseError 2012-02-23 15:42 [PATCH 0/2] A couple of fixes to error handling Paul Eggleton 2012-02-23 15:42 ` [PATCH 1/2] bitbake: fix parse errors not being reported Paul Eggleton @ 2012-02-23 15:42 ` Paul Eggleton 1 sibling, 0 replies; 6+ messages in thread From: Paul Eggleton @ 2012-02-23 15:42 UTC (permalink / raw) To: bitbake-devel Ensure that a file and line number are reported for ParseError where possible. This helps particularly in the case of inherit and require which previously did not report either of these upon failure. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> --- bitbake/lib/bb/cooker.py | 2 +- bitbake/lib/bb/parse/__init__.py | 15 +++++++++++++-- bitbake/lib/bb/parse/ast.py | 8 ++++---- bitbake/lib/bb/parse/parse_py/BBHandler.py | 6 ++---- bitbake/lib/bb/parse/parse_py/ConfHandler.py | 11 ++++++----- 5 files changed, 26 insertions(+), 16 deletions(-) diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index 5653874..8e30cea 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -1398,7 +1398,7 @@ def _parse(fn, data, include=True): @catch_parse_error def _inherit(bbclass, data): - bb.parse.BBHandler.inherit([bbclass], data) + bb.parse.BBHandler.inherit([bbclass], "configuration INHERITs", 0, data) return data class ParsingFailure(Exception): diff --git a/bitbake/lib/bb/parse/__init__.py b/bitbake/lib/bb/parse/__init__.py index c5005ae..8b7ec73 100644 --- a/bitbake/lib/bb/parse/__init__.py +++ b/bitbake/lib/bb/parse/__init__.py @@ -37,6 +37,17 @@ logger = logging.getLogger("BitBake.Parsing") class ParseError(Exception): """Exception raised when parsing fails""" + def __init__(self, msg, filename, lineno=0): + self.msg = msg + self.filename = filename + self.lineno = lineno + Exception.__init__(self, msg, filename, lineno) + + def __str__(self): + if self.lineno: + return "ParseError at %s:%d: %s" % (self.filename, self.lineno, self.msg) + else: + return "ParseError in %s: %s" % (self.filename, self.msg) class SkipPackage(Exception): """Exception raised to skip this package""" @@ -78,7 +89,7 @@ def handle(fn, data, include = 0): for h in handlers: if h['supports'](fn, data): return h['handle'](fn, data, include) - raise ParseError("%s is not a BitBake file" % fn) + raise ParseError("not a BitBake file", fn) def init(fn, data): for h in handlers: @@ -111,7 +122,7 @@ def vars_from_file(mypkg, d): parts = myfile[0].split('_') __pkgsplit_cache__[mypkg] = parts if len(parts) > 3: - raise ParseError("Unable to generate default variables from the filename: %s (too many underscores)" % mypkg) + raise ParseError("Unable to generate default variables from filename (too many underscores)", mypkg) exp = 3 - len(parts) tmplist = [] while exp != 0: diff --git a/bitbake/lib/bb/parse/ast.py b/bitbake/lib/bb/parse/ast.py index 31c930d..94fa175 100644 --- a/bitbake/lib/bb/parse/ast.py +++ b/bitbake/lib/bb/parse/ast.py @@ -59,9 +59,9 @@ class IncludeNode(AstNode): # TODO: Cache those includes... maybe not here though if self.force: - bb.parse.ConfHandler.include(self.filename, s, data, "include required") + bb.parse.ConfHandler.include(self.filename, s, self.lineno, data, "include required") else: - bb.parse.ConfHandler.include(self.filename, s, data, False) + bb.parse.ConfHandler.include(self.filename, s, self.lineno, data, False) class ExportNode(AstNode): def __init__(self, filename, lineno, var): @@ -267,7 +267,7 @@ class InheritNode(AstNode): self.classes = classes def eval(self, data): - bb.parse.BBHandler.inherit(self.classes, data) + bb.parse.BBHandler.inherit(self.classes, self.filename, self.lineno, data) def handleInclude(statements, filename, lineno, m, force): statements.append(IncludeNode(filename, lineno, m.group(1), force)) @@ -450,7 +450,7 @@ def multi_finalize(fn, d): d.setVar("BBEXTENDVARIANT", variantmap[name]) else: d.setVar("PN", "%s-%s" % (pn, name)) - bb.parse.BBHandler.inherit([extendedmap[name]], d) + bb.parse.BBHandler.inherit([extendedmap[name]], fn, 0, d) safe_d.setVar("BBCLASSEXTEND", extended) _create_variants(datastores, extendedmap.keys(), extendfunc) diff --git a/bitbake/lib/bb/parse/parse_py/BBHandler.py b/bitbake/lib/bb/parse/parse_py/BBHandler.py index 2d6e331..125f458 100644 --- a/bitbake/lib/bb/parse/parse_py/BBHandler.py +++ b/bitbake/lib/bb/parse/parse_py/BBHandler.py @@ -68,10 +68,8 @@ def supports(fn, d): """Return True if fn has a supported extension""" return os.path.splitext(fn)[-1] in [".bb", ".bbclass", ".inc"] -def inherit(files, d): +def inherit(files, fn, lineno, d): __inherit_cache = data.getVar('__inherit_cache', d) or [] - fn = "" - lineno = 0 for file in files: file = data.expand(file, d) if not os.path.isabs(file) and not file.endswith(".bbclass"): @@ -81,7 +79,7 @@ def inherit(files, d): logger.log(logging.DEBUG -1, "BB %s:%d: inheriting %s", fn, lineno, file) __inherit_cache.append( file ) data.setVar('__inherit_cache', __inherit_cache, d) - include(fn, file, d, "inherit") + include(fn, file, lineno, d, "inherit") __inherit_cache = data.getVar('__inherit_cache', d) or [] def get_statements(filename, absolute_filename, base_name): diff --git a/bitbake/lib/bb/parse/parse_py/ConfHandler.py b/bitbake/lib/bb/parse/parse_py/ConfHandler.py index 6ae9d97..9242632 100644 --- a/bitbake/lib/bb/parse/parse_py/ConfHandler.py +++ b/bitbake/lib/bb/parse/parse_py/ConfHandler.py @@ -44,10 +44,11 @@ def init(data): def supports(fn, d): return fn[-5:] == ".conf" -def include(oldfn, fn, data, error_out): +def include(oldfn, fn, lineno, data, error_out): """ - error_out If True a ParseError will be raised if the to be included - config-files could not be included. + error_out: A string indicating the verb (e.g. "include", "inherit") to be + used in a ParseError that will be raised if the file to be included could + not be included. Specify False to avoid raising an error in this case. """ if oldfn == fn: # prevent infinite recursion return None @@ -68,7 +69,7 @@ def include(oldfn, fn, data, error_out): ret = handle(fn, data, True) except IOError: if error_out: - raise ParseError("Could not %(error_out)s file %(fn)s" % vars() ) + raise ParseError("Could not %(error_out)s file %(fn)s" % vars(), oldfn, lineno) logger.debug(2, "CONF file '%s' not found", fn) def handle(fn, data, include): @@ -131,7 +132,7 @@ def feeder(lineno, s, fn, statements): ast.handleExport(statements, fn, lineno, m) return - raise ParseError("%s:%d: unparsed line: '%s'" % (fn, lineno, s)); + raise ParseError("unparsed line: '%s'" % s, fn, lineno); # Add us to the handlers list from bb.parse import handlers -- 1.7.5.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-02-23 17:35 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-02-23 15:42 [PATCH 0/2] A couple of fixes to error handling Paul Eggleton 2012-02-23 15:42 ` [PATCH 1/2] bitbake: fix parse errors not being reported Paul Eggleton 2012-02-23 16:14 ` Chris Larson 2012-02-23 17:23 ` Paul Eggleton 2012-02-23 17:26 ` Chris Larson 2012-02-23 15:42 ` [PATCH 2/2] bitbake: add file and line number to ParseError Paul Eggleton
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox