Re: [OE-core] bb.utils.vercmp_string_op() vs Python 3.12

["Böszörményi Zoltán" <zboszor@gmail.com>]

2024. 01. 26. 11:53 keltezéssel, Martin Jansa írta:
> > I copied vercmp_string_op() and all others called by it from bitbake/lib/bb/utils.py 
> into a new python script.
>
> It's not an issue in vercmp_string_op itself,

I already know this.
Apparently you haven't read my mail fully for the conclusion.

> but the version value it gets from waf --version isn't just the version itself but also 
> SyntaxWarning from waf.

That is not a problem in itself as the preceding call only collects the stdout:

result = subprocess.check_output([python, wafbin, '--version'], cwd=subsrcdir, 
stderr=subprocess.STDOUT)

SyntaxWarning goes (or should go) to stderr.

When this part is moved into a minimal test case, there is no error at all.

The error only happens inside bitbake and there was no such problem
with Python 3.11.x or earlier.

Ergo, the problem must be a Python 3.12 issue.
The actual version here is python3-3.12.1-2.fc39.x86_64 for me.

> So you need to update waf in jack not to trigger SyntaxWarning when waf --version is called.
>
> On Fri, Jan 26, 2024 at 10:17 AM Zoltan Boszormenyi <zboszor@gmail.com> wrote:
>
>     Hi,
>
>     Just a FYI.
>
>     After the bitbake-server getting stuck after bitbake exited
>     with Python 3.12 issue was fixed, here's another one.
>
>     This time it's semi-reliably triggered by the jack recipe
>     in meta-openembedded:
>
>     ERROR: jack-1.19.22-r0 do_configure: Error executing a python function in
>     exec_func_python() autogenerated:
>
>     The stack trace of python calls that resulted in this exception/failure was:
>     File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>           0001:
>       *** 0002:waf_preconfigure(d)
>           0003:
>     File:
>     '/home/zozo/test-yocto-4.3-gh/conf/../layers/openembedded-core/meta/classes-recipe/waf.bbclass',
>
>     lineno: 58, function: waf_preconfigure
>           0054:    wafbin = os.path.join(subsrcdir, 'waf')
>           0055:    try:
>           0056:        result = subprocess.check_output([python, wafbin, '--version'],
>     cwd=subsrcdir, stderr=subprocess.STDOUT)
>           0057:        version = result.decode('utf-8').split()[1]
>       *** 0058:        if bb.utils.vercmp_string_op(version, "1.8.7", ">="):
>           0059:            d.setVar("WAF_EXTRA_CONF", "--bindir=${bindir}
>     --libdir=${libdir}")
>           0060:    except subprocess.CalledProcessError as e:
>           0061:        bb.warn("Unable to execute waf --version, exit code %d. Assuming waf
>     version without bindir/libdir support." % e.returncode)
>           0062:    except FileNotFoundError:
>     File: '/home/zozo/test-yocto-4.3-gh/bitbake/lib/bb/utils.py', lineno: 150, function:
>     vercmp_string_op
>           0146:    Compare two versions and check if the specified comparison operator
>     matches
>     the result of the comparison.
>           0147:    This function is fairly liberal about what operators it will accept
>     since
>     there are a variety of styles
>           0148:    depending on the context.
>           0149:    """
>       *** 0150:    res = vercmp_string(a, b)
>           0151:    if op in ('=', '=='):
>           0152:        return res == 0
>           0153:    elif op == '<=':
>           0154:        return res <= 0
>     File: '/home/zozo/test-yocto-4.3-gh/bitbake/lib/bb/utils.py', lineno: 140, function:
>     vercmp_string
>           0136:    return r
>           0137:
>           0138:def vercmp_string(a, b):
>           0139:    """ Split version strings and compare them """
>       *** 0140:    ta = split_version(a)
>           0141:    tb = split_version(b)
>           0142:    return vercmp(ta, tb)
>           0143:
>           0144:def vercmp_string_op(a, b, op):
>     File: '/home/zozo/test-yocto-4.3-gh/bitbake/lib/bb/utils.py', lineno: 91, function:
>     split_version
>           0087:    """Split a version string into its constituent parts (PE, PV, PR)"""
>           0088:    s = s.strip(" <>=")
>           0089:    e = 0
>           0090:    if s.count(':'):
>       *** 0091:        e = int(s.split(":")[0])
>           0092:        s = s.split(":")[1]
>           0093:    r = ""
>           0094:    if s.count('-'):
>           0095:        r = s.rsplit("-", 1)[1]
>     Exception: ValueError: invalid literal for int() with base 10: 'SyntaxWarning'
>
>     ERROR: Logfile of failure stored in:
>     /home/zozo/test-yocto-4.3-gh/tmp-sicom-glibc/work/corei7-64-oe-linux/jack/1.19.22/temp/log.do_configure.1709417
>
>     "Semi-reliably", i.e. about 50-50 or 60-40 percent failure vs success rate.
>     Usually the second run succeeds after a failure.
>
>     I copied vercmp_string_op() and all others called by it
>     from bitbake/lib/bb/utils.py into a new python script as a
>     minimum test case. Running this script inside a locally
>     cloned jack git repository has 100% success rate.
>
>     It looks like an internal Python "flake" issue.
>
>     Best regards,
>     Zoltán Böszörményi
>
>
>     -=-=-=-=-=-=-=-=-=-=-=-
>     Links: You receive all messages sent to this group.
>     View/Reply Online (#194348):
>     https://lists.openembedded.org/g/openembedded-core/message/194348
>     Mute This Topic: https://lists.openembedded.org/mt/103973063/3617156
>     Group Owner: openembedded-core+owner@lists.openembedded.org
>     <mailto:openembedded-core%2Bowner@lists.openembedded.org>
>     Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>     [martin.jansa@gmail.com]
>     -=-=-=-=-=-=-=-=-=-=-=-
>