Re: [OE-core] [PATCH 19/24] python3-setuptools: upgrade 72.1.0 -> 72.2.0

[Trevor Gamblin <tgamblin@baylibre.com>]

On 2024-08-22 2:26 p.m., Martin Jansa via lists.openembedded.org wrote:
> It seems to fail only when ccache is enabled, I'll send patch to add
> CCACHE_DISABLE to numpy recipe to work around this issue for now.

Even with that workaround, I see a more serious issue:

|DEBUG: Executing shell function do_compile
|/home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/1.26.4/numpy-1.26.4/setup.py:60: 
RuntimeWarning: NumPy 1.26.4 may not yet support Python 3.12.
| warnings.warn(
|Running from numpy source directory.
|/home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/1.26.4/numpy-1.26.4/setup.py:75: 
DeprecationWarning:
|
| `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
|  of the deprecation of `distutils` itself. It will be removed for
|  Python >= 3.12. For older Python versions it will remain present.
|  It is recommended to use `setuptools < 60.0` for those Python versions.
|  For more details, see:
| https://numpy.org/devdocs/reference/distutils_status_migration.html
|
|
|  import numpy.distutils.command.sdist
|Traceback (most recent call last):
|  File 
"/home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/1.26.4/numpy-1.26.4/setup.py", 
line 483, in <module>
|    from numpy.distutils.core import numpy_cmdclass as cmdclass
|  File 
"/home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/1.26.4/numpy-1.26.4/numpy/distutils/core.py", 
line 24, in <module>
|    from numpy.distutils.command import config, config_compiler, \
|  File 
"/home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/1.26.4/numpy-1.26.4/numpy/distutils/command/config.py", 
line 19, in <module>
|    from numpy.distutils.mingw32ccompiler import generate_manifest
|  File 
"/home/tgamblin/workspace/yocto/poky/build/tmp/work/core2-64-poky-linux/python3-numpy/1.26.4/numpy-1.26.4/numpy/distutils/mingw32ccompiler.py", 
line 27, in <module>
|    from distutils.msvccompiler import get_build_version as 
get_build_msvc_version
|ModuleNotFoundError: No module named 'distutils.msvccompiler'
|ERROR: 'python3 setup.py bdist_wheel ' execution failed.

I think we'll need to wait for the numpy upgrade to go in.

>
> On Thu, Aug 22, 2024 at 6:30 PM Martin Jansa via
> lists.openembedded.org <martin.jansa=gmail.com@lists.openembedded.org>
> wrote:
>> This seems to break python3-numpy build.
>>
>> I guess it can be resolved by upgrading numpy as in:
>> https://lists.openembedded.org/g/openembedded-core/message/203231
>> which switches from setuptools3 to python_mesonpy
>>
>> Comparing the failing command line it looks like CXX and some other
>> flags are included twice:
>>
>> with: setuptools-72.2.0:
>>
>> ccache \
>> aarch64-oe-linux-g++ \
>> -fstack-protector-all \
>> -O2 \
>> -D_FORTIFY_SOURCE=2 \
>> -Wformat \
>> -Wformat-security \
>> -Werror=format-security \
>> -Werror=return-type \
>> -funwind-tables \
>> --sysroot=WORKDIR/recipe-sysroot \
>> -O2 \
>> -g \
>> -fcanon-prefix-map \
>> -fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/recipe-sysroot= \
>> -fmacro-prefix-map=WORKDIR/recipe-sysroot= \
>> -fdebug-prefix-map=WORKDIR/recipe-sysroot-native= \
>> -pipe \
>> -fvisibility-inlines-hidden \
>> aarch64-oe-linux-g++ \
>> -fstack-protector-all \
>> -O2 \
>> -D_FORTIFY_SOURCE=2 \
>> -Wformat \
>> -Wformat-security \
>> -Werror=format-security \
>> -Werror=return-type \
>> -funwind-tables \
>> --sysroot=WORKDIR/recipe-sysroot \
>> -shared \
>> -Wl,-O1 \
>> -Wl,--hash-style=gnu \
>> -Wl,--as-needed \
>> -fcanon-prefix-map \
>> -fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/recipe-sysroot= \
>> -fmacro-prefix-map=WORKDIR/recipe-sysroot= \
>> -fdebug-prefix-map=WORKDIR/recipe-sysroot-native= \
>> -Wl,-z,relro,-z,now \
>> -O2 \
>> -g \
>> -fcanon-prefix-map \
>> -fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> -fdebug-prefix-map=WORKDIR/recipe-sysroot= \
>> -fmacro-prefix-map=WORKDIR/recipe-sysroot= \
>> -fdebug-prefix-map=WORKDIR/recipe-sysroot-native= \
>> -pipe \
>> -fvisibility-inlines-hidden \
>> build/temp.linux-x86_64-cpython-312/build/src.linux-x86_64-3.12/numpy/core/src/multiarray/_multiarray_tests.o
>> \
>> build/temp.linux-x86_64-cpython-312/numpy/core/src/common/mem_overlap.o \
>> build/temp.linux-x86_64-cpython-312/numpy/core/src/common/npy_argparse.o \
>> build/temp.linux-x86_64-cpython-312/numpy/core/src/common/npy_hashtable.o \
>> -LWORKDIR/recipe-sysroot/usr/lib \
>> -Lbuild/temp.linux-x86_64-cpython-312 \
>> -lnpymath \
>> -o \
>> build/lib.linux-x86_64-cpython-312/numpy/core/_multiarray_tests.cpython-312-aarch64-linux-gnu.so
>>
>> --- setuptools-72.1.0   2024-08-22 18:17:36.941146726 +0200
>> +++ setuptools-72.2.0   2024-08-22 18:17:27.208147768 +0200
>> @@ -9,6 +9,28 @@
>>   -Werror=return-type \
>>   -funwind-tables \
>>   --sysroot=WORKDIR/recipe-sysroot \
>> +-O2 \
>> +-g \
>> +-fcanon-prefix-map \
>> +-fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> +-fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> +-fmacro-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> +-fdebug-prefix-map=WORKDIR/numpy-1.26.4=/usr/src/debug/python3-numpy/1.26.4 \
>> +-fdebug-prefix-map=WORKDIR/recipe-sysroot= \
>> +-fmacro-prefix-map=WORKDIR/recipe-sysroot= \
>> +-fdebug-prefix-map=WORKDIR/recipe-sysroot-native= \
>> +-pipe \
>> +-fvisibility-inlines-hidden \
>> +aarch64-oe-linux-g++ \
>> +-fstack-protector-all \
>> +-O2 \
>> +-D_FORTIFY_SOURCE=2 \
>> +-Wformat \
>> +-Wformat-security \
>> +-Werror=format-security \
>> +-Werror=return-type \
>> +-funwind-tables \
>> +--sysroot=WORKDIR/recipe-sysroot \
>>   -shared \
>>   -Wl,-O1 \
>>   -Wl,--hash-style=gnu \
>> @@ -33,6 +55,7 @@
>>   -fmacro-prefix-map=WORKDIR/recipe-sysroot= \
>>   -fdebug-prefix-map=WORKDIR/recipe-sysroot-native= \
>>   -pipe \
>> +-fvisibility-inlines-hidden \
>>   build/temp.linux-x86_64-cpython-312/build/src.linux-x86_64-3.12/numpy/core/src/multiarray/_multiarray_tests.o
>> \
>>   build/temp.linux-x86_64-cpython-312/numpy/core/src/common/mem_overlap.o \
>>   build/temp.linux-x86_64-cpython-312/numpy/core/src/common/npy_argparse.o \
>>
>> failing with:
>> WORKDIR/recipe-sysroot-native/usr/bin/aarch64-oe-linux/../../libexec/aarch64-oe-linux/gcc/aarch64-oe-linux/14.2.0/ld:
>> cannot find aarch64-oe-linux-g++: No such file or directory
>> collect2: error: ld returned 1 exit status
>>
>> On Tue, Aug 20, 2024 at 8:52 AM wangmy via lists.openembedded.org
>> <wangmy=fujitsu.com@lists.openembedded.org> wrote:
>>> From: Wang Mingyu <wangmy@fujitsu.com>
>>>
>>> Changelog:
>>> - Support for Pathlike objects in data files and extensions
>>> - native support for C++ compilers
>>> - removed unused get_msvcr()
>>>
>>> Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
>>> ---
>>>   ...ython3-setuptools_72.1.0.bb => python3-setuptools_72.2.0.bb} | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>   rename meta/recipes-devtools/python/{python3-setuptools_72.1.0.bb => python3-setuptools_72.2.0.bb} (94%)
>>>
>>> diff --git a/meta/recipes-devtools/python/python3-setuptools_72.1.0.bb b/meta/recipes-devtools/python/python3-setuptools_72.2.0.bb
>>> similarity index 94%
>>> rename from meta/recipes-devtools/python/python3-setuptools_72.1.0.bb
>>> rename to meta/recipes-devtools/python/python3-setuptools_72.2.0.bb
>>> index 945d443aff..63ea999960 100644
>>> --- a/meta/recipes-devtools/python/python3-setuptools_72.1.0.bb
>>> +++ b/meta/recipes-devtools/python/python3-setuptools_72.2.0.bb
>>> @@ -11,7 +11,7 @@ SRC_URI:append:class-native = " file://0001-conditionally-do-not-fetch-code-by-e
>>>   SRC_URI += " \
>>>               file://0001-_distutils-sysconfig.py-make-it-possible-to-substite.patch"
>>>
>>> -SRC_URI[sha256sum] = "8d243eff56d095e5817f796ede6ae32941278f542e0f941867cc05ae52b162ec"
>>> +SRC_URI[sha256sum] = "80aacbf633704e9c8bfa1d99fa5dd4dc59573efcf9e4042c13d3bcef91ac2ef9"
>>>
>>>   DEPENDS += "python3"
>>>
>>> --
>>> 2.34.1
>>>
>>>
>>>
>>>
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#203661): https://lists.openembedded.org/g/openembedded-core/message/203661
>> Mute This Topic: https://lists.openembedded.org/mt/107996505/7611679
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [tgamblin@baylibre.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>