Weird python/OE issue on parsing

Weird python/OE issue on parsing

[Holger Freyther]

Hi,

my build to track master of Poky has started to fail in a very odd way. I can't reproduce it in a
local build but something is severely wrong:

ERROR: Error in compiling python function in /home/oebuilds/jenkins/workspace/Yocto-Master/label/OE/poky/meta/conf/distro/include/tclibc-glibc.inc, line -10000000000005:
..
OverflowError: Python int too large to convert to C long


So somehow IN_PYTHON_EOF starts ending up in linenumber and then a bit is subtracted from
it. Any idea how I can run into this?

holger

Re: Weird python/OE issue on parsing

[Holger Freyther]

> On 19 Dec 2015, at 13:15, Holger Freyther <holger@freyther.de> wrote:
> 
> So somehow IN_PYTHON_EOF starts ending up in linenumber and then a bit is subtracted from
> it. Any idea how I can run into this?

The error is gone when I revert 0019edc8180e859544da4820fa390b23d975fe08 in bitbake. I can
re-produce this with a debootstrap wheezy/i386 and setting up my build.

holger

Re: Weird python/OE issue on parsing

[Richard Purdie]

On Sat, 2015-12-19 at 13:15 +0100, Holger Freyther wrote:
> Hi,
> 
> my build to track master of Poky has started to fail in a very odd
> way. I can't reproduce it in a
> local build but something is severely wrong:
> 
> ERROR: Error in compiling python function in
> /home/oebuilds/jenkins/workspace/Yocto
> -Master/label/OE/poky/meta/conf/distro/include/tclibc-glibc.inc, line
> -10000000000005:
> ..
> OverflowError: Python int too large to convert to C long
> 
> 
> So somehow IN_PYTHON_EOF starts ending up in linenumber and then a
> bit is subtracted from
> it. Any idea how I can run into this?

It certainly seems related to the line numbering changes maybe
interacting badly with EOF somehow. Is this on a 32 bit system? 

Cheers,

Richard

Re: Weird python/OE issue on parsing

[Richard Purdie]

On Sat, 2015-12-19 at 23:02 +0000, Richard Purdie wrote:
> On Sat, 2015-12-19 at 13:15 +0100, Holger Freyther wrote:
> > Hi,
> > 
> > my build to track master of Poky has started to fail in a very odd
> > way. I can't reproduce it in a
> > local build but something is severely wrong:
> > 
> > ERROR: Error in compiling python function in
> > /home/oebuilds/jenkins/workspace/Yocto
> > -Master/label/OE/poky/meta/conf/distro/include/tclibc-glibc.inc,
> > line
> > -10000000000005:
> > ..
> > OverflowError: Python int too large to convert to C long
> > 
> > 
> > So somehow IN_PYTHON_EOF starts ending up in linenumber and then a
> > bit is subtracted from
> > it. Any idea how I can run into this?
> 
> It certainly seems related to the line numbering changes maybe
> interacting badly with EOF somehow. Is this on a 32 bit system? 

Putting:

+        if "tclibc-glibc" in realfile:
+            bb.warn(str(lineno))

into better_compile() shows up some nasty looking line numbers. I think
we need to better bounds checks on the line numbers and fix some of the
data that is being passed in for functions at EOF.

Cheers,

Richard

Re: Weird python/OE issue on parsing

[Holger Freyther]

> On 20 Dec 2015, at 00:02, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> 
> It certainly seems related to the line numbering changes maybe
> interacting badly with EOF somehow. Is this on a 32 bit system? 

Yes, that is a 32bit userspace. debootstrap --arch=i386 wheezy wheezy and then
installing the deps and a plain git.yoctoproject,org/poky checkout is enough to see
the issue.

holger