Re: [PATCH] utils: Remove double compile from better_compile

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2016-01-04 at 11:06 -0700, Christopher Larson wrote:
> 
> On Mon, Jan 4, 2016 at 10:34 AM, Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
> > Poking around the ast to correct linenumbers works well for runtime
> > failures
> > but not for parsing ones. We can use blank linefeeds to correct the
> > line
> > numbers instead, with the advantage that we don't need to double
> > compile.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > 
> > diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
> > index cd5fced..9a3efb2 100644
> > --- a/bitbake/lib/bb/utils.py
> > +++ b/bitbake/lib/bb/utils.py
> > @@ -292,7 +292,7 @@ def _print_trace(body, line):
> >              error.append('     %.4d:%s' % (i, body[i-1].rstrip()))
> >      return error
> > 
> > -def better_compile(text, file, realfile, mode = "exec", lineno =
> > None):
> > +def better_compile(text, file, realfile, mode = "exec", lineno =
> > 0):
> >      """
> >      A better compile method. This method
> >      will print the offending lines.
> > @@ -301,10 +301,9 @@ def better_compile(text, file, realfile, mode
> > = "exec", lineno = None):
> >          cache = bb.methodpool.compile_cache(text)
> >          if cache:
> >              return cache
> > -        code = compile(text, realfile, mode, ast.PyCF_ONLY_AST)
> > -        if lineno is not None:
> > -            ast.increment_lineno(code, lineno)
> > -        code = compile(code, realfile, mode)
> > +        # We can't add to the linenumbers for compile, we can pad
> > to the correct number of blank lines though
> > +        text2 = "\n" * int(lineno) + text
> > +        code = compile(text2, realfile, mode)
> > 
> A SyntaxError can have its line numbers adjusted fairly easily, if
> that's your concern. Afaik that's the usual case which
> ast.increment_lineno isn't  sufficient to handle. See 
> https://gist.github.com/kergoth/743677#file-compile-py-L67-L86.

I did start to wonder about catching specific exceptions and then
adjusting the line numbers. It seemed unlikely that we'd catch all the
right cases and adjust them correctly though and when there is a much
easier workaround with a few blank lines that likely performs better
than the existing double compile, I concluded that whilst not that
architecturally pleasing, it is simple and effective...

Cheers,

Richard