Re: [OE-core][PATCH 0/5 v2] python3: upgrade 3.13.9 -> 3.14.2

[Trevor Gamblin <tgamblin@baylibre.com>]

On 2025-12-22 03:11, Mathieu Dubois-Briand wrote:
> On Fri Dec 19, 2025 at 2:47 PM CET, Trevor Gamblin wrote:
>> Compared to v1
>> (https://lists.openembedded.org/g/openembedded-core/message/227955):
>>
>> - Drop python3-bcrypt upgrade, as someone else has submitted that;
>> - Drop rpm patch to manually override Python3_SITEARCH with a patch, as that was
>>    masking the real issue (see below);
>> - Add a new patch to fix python3targetconfig by exporting _PYTHON_PROJECT_BASE,
>>    which Python's sysconfig looks for (as of 3.14.0) when doing
>>    cross-compilation. The absence of this was resulting in packages like rpm
>>    finding paths for the native interpreter when making use of sysconfig, when
>>    they needed the host values.
>>
>> Note that I've dropped my own version of the python3-bcrypt upgrade, but I
>> tested this patch series on top of the other one that was submitted, as
>> otherwise the ptest images would break during build (version mismatch errors
>> around PyO3, which is fixed with the bcrypt upgrade).
>>
>> v1 details below.
>>
>> This supersedes a previous series sent, which only did the 3.14.0 upgrade.
>>
>> Compared to the first series
>> (https://lists.openembedded.org/g/openembedded-core/message/227310), I've
>> done the following:
>>
>> - Added a commit for including the '_py_warnings' and 'annotationlib' modules in
>>    python3-core;
>> - Adds the python 3.14.2 upgrade;
>> - Removed commits for adding python3-misc to RDEPENDS for python3-pytest and
>>    python3-jsonpointer, as these are now solved by a new patch to add new modules
>>    to python3-core in the manifest. Doing it this way was fragile in the sense
>>    that there were many other recipes needing this same fix that testing didn't
>>    easily catch, and it doesn't make sense to add python3-misc to recipe RDEPENDS
>>    just because (for example) python3-logging (another package split) is trying
>>    to make use of warnings, but can't find the right module;
>> - Modified the rpm commit to apply a patch (submitted upstream) to fix
>>    CMakeLists.txt, rather than a do_configure prepend;
>> - Fixed the Python 3.14.0 upgrade commit so that one of the patches we carry to
>>    skip tests doesn't include duplicate calls to the relevant skip functions.
>>
>> I've kept the upgrades from 3.13.9 to 3.14.0 and 3.14.0 to 3.14.2 separate to
>> reflect separate testing and let people review those changelogs individually if
>> desired.
>>
>> Trevor Gamblin (5):
>>    python3: upgrade 3.13.9 -> 3.14.0
>>    python3targetconfig: export _PYTHON_PROJECT_BASE
>>    python_pep517: add --prefix to nativepython3 call
>>    python3: upgrade 3.14.0 -> 3.14.2
>>    python3: add _py_warnings, annotationlib to core
>>
> Hi Trevor,
>
> Thanks for the new version. It looks like we are having issues with
> maturin and click.
>
> Testing 'maturin develop' requires Rust cross-canadian in the SDK
> Traceback (most recent call last):
>    File "/srv/pokybuild/yocto-worker/qemux86/build/layers/openembedded-core/meta/lib/oeqa/sdk/cases/maturin.py", line 30, in test_maturin_list_python
>      self.assertIn(expected, output)
>      ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
> AssertionError: '3.14 at /srv/pokybuild/yocto-worker/qemux86/build/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0/testsdkext/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3' not found in '🐍 1 python interpreter found:\n - CPython 3.13 at /srv/pokybuild/yocto-worker/qemux86/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3.13\n'
> ...
> RESULTS - maturin.MaturinTest.test_maturin_list_python: FAILED (3.57s)
I'm not sure how to address this one - it looks like the problem is that 
the buildtools tarball has a 3.13.4 binary in it (at least when I try 
locally). Where does that get updated?
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/30/builds/2876
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/2900
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/45/builds/837
>
> AssertionError:
> Failed ptests:
> {'python3-click': 'START: ptest-runner\n'
>                    '2025-12-21T20:15\n'
>                    '\n'
>                    '../../python3.14/ast.py:620: in unparse\n'
>                    '    unparser = _Unparser()\n'
>                    '               ^^^^^^^^^\n'
>                    "E   NameError: name '_Unparser' is not defined\n"
>                    '\n'
>                    'During handling of the above exception, another exception '
>                    'occurred:\n'
>                    '../../python3.14/site-packages/pluggy/_hooks.py:512: in '
>                    '__call__\n'
>                    '    return self._hookexec(self.name, '
>                    'self._hookimpls.copy(), kwargs, firstresult)\n'
>                    '           '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/pluggy/_manager.py:120: in '
>                    '_hookexec\n'
>                    '    return self._inner_hookexec(hook_name, methods, kwargs, '
>                    'firstresult)\n'
>                    '           '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/python.py:240: in '
>                    'pytest_pycollect_makeitem\n'
>                    '    return list(collector._genfunctions(name, obj))\n'
>                    '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/python.py:448: in '
>                    '_genfunctions\n'
>                    '    definition = FunctionDefinition.from_parent(self, '
>                    'name=name, callobj=funcobj)\n'
>                    '                 '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/python.py:1625: in '
>                    'from_parent\n'
>                    '    return super().from_parent(parent=parent, **kw)\n'
>                    '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/nodes.py:233: in '
>                    'from_parent\n'
>                    '    return cls._create(parent=parent, **kw)\n'
>                    '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/nodes.py:110: in '
>                    '_create\n'
>                    '    return super().__call__(*k, **kw)  # type: '
>                    'ignore[no-any-return,misc]\n'
>                    '           ^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/python.py:1616: in '
>                    '__init__\n'
>                    '    fixtureinfo = fm.getfixtureinfo(self, self.obj, '
>                    'self.cls)\n'
>                    '                  '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/fixtures.py:1565: in '
>                    'getfixtureinfo\n'
>                    '    argnames = getfuncargnames(func, name=node.name, '
>                    'cls=cls)\n'
>                    '               '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/compat.py:136: in '
>                    'getfuncargnames\n'
>                    '    parameters = signature(function).parameters.values()\n'
>                    '                 ^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/site-packages/_pytest/compat.py:70: in '
>                    'signature\n'
>                    '    return inspect.signature(obj, '
>                    'annotation_format=Format.STRING)\n'
>                    '           '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/inspect.py:3321: in signature\n'
>                    '    return Signature.from_callable(obj, '
>                    'follow_wrapped=follow_wrapped,\n'
>                    '../../python3.14/inspect.py:3036: in from_callable\n'
>                    '    return _signature_from_callable(obj, sigcls=cls,\n'
>                    '../../python3.14/inspect.py:2511: in '
>                    '_signature_from_callable\n'
>                    '    return _signature_from_function(sigcls, obj,\n'
>                    '../../python3.14/inspect.py:2334: in '
>                    '_signature_from_function\n'
>                    '    annotations = get_annotations(func, globals=globals, '
>                    'locals=locals, eval_str=eval_str,\n'
>                    '../../python3.14/annotationlib.py:981: in get_annotations\n'
>                    '    ann = _get_and_call_annotate(obj, format)\n'
>                    '          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/annotationlib.py:1112: in '
>                    '_get_and_call_annotate\n'
>                    '    ann = call_annotate_function(annotate, format, '
>                    'owner=obj)\n'
>                    '          '
>                    '^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/annotationlib.py:750: in '
>                    'call_annotate_function\n'
>                    '    key: _stringify_single(val)\n'
>                    '         ^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/annotationlib.py:883: in '
>                    '_stringify_single\n'
>                    '    return repr(anno)\n'
>                    '           ^^^^^^^^^^\n'
>                    '../../python3.14/annotationlib.py:487: in __repr__\n'
>                    '    return ast.unparse(self.__ast_node__)\n'
>                    '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
>                    '../../python3.14/ast.py:622: in unparse\n'
>                    '    from _ast_unparse import Unparser as _Unparser\n'
>                    "E   ModuleNotFoundError: No module named '_ast_unparse'\n"
>                    'ERROR: '
>                    'tests/test_shell_completion.py:tests/test_shell_completion.py\n'
>                    '============================================================================\n'
>                    'Testsuite summary\n'
>                    '# TOTAL: 1\n'
>                    '# PASS: 0\n'
>                    '# SKIP: 0\n'
>                    '# XFAIL: 0\n'
>                    '# FAIL: 0\n'
>                    '# XPASS: 0\n'
>                    '# ERROR: 1\n'
>                    '\n'
>                    'ERROR: Exit status is 2\n'
>                    'DURATION: 28\n'}
>
> ptests which had no test results:
> ['python3-click']
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/860
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/2751
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/2794
>
> Ptest logs can be found here:
>
> http://valkyrie.yocto.io/pub/non-release/20251221-102/testresults/qemuriscv64-ptest/core-image-ptest-python3-click/
> http://valkyrie.yocto.io/pub/non-release/20251221-102/testresults/qemuarm64-ptest/core-image-ptest-python3-click/
> http://valkyrie.yocto.io/pub/non-release/20251221-102/testresults/qemux86-64-ptest/core-image-ptest-python3-click/
>
> Can you have a look at these failures?
>
> Thanks,
> Mathieu
>