[RFC][PATCH 00/14] Support Python freethreading

[RFC][PATCH 00/14] Support Python freethreading

[Zoltán Böszörményi]

This patchset allows turning on Python freethreading by using
a new DISTRO_FEATURES setting:

DISTRO_FEATURES += "python_freethreading"

and optionally:

DISTRO_FEATURES_NATIVE += "python_freethreading"

The change to python3-dir.bbclass is crucial, as all python module
build helper classes rely on it.

I also added native variants of some settings because another
series against meta-openembedded is needed to complete this one.
In that, the change to the fontforge recipe relies on the native
setting for setting BUILD_LDFLAGS.

All of the changes included here were successfully build tested,
except for the gdb-cross-canadian change. That one failed, but
it's included for completeness.

Quite a few of python modules were build tested for native and
target builds, including python3-pytorch from my meta-python-ai
with reduced RDEPENDS because e.g. python3-onnx there failed
to build with freethreading enabled.

All in all, this series at least allows to get things going.
Though maybe not for Yocto 6.0, as it's quite late.

Best regards,
Zoltán Böszörményi

[RFC][PATCH 01/14] python3-dir.bbclass: Support python freethreading

[Zoltán Böszörményi]

With a new DISTRO_FEATURE ("python_freethreading"), set PYTHON_ABI.

Added a new variable called PYTHON_MAINVERSION which is a
combination of PYTHON_BASEVERSION and PYTHON_ABI.

Set PYTHON_DIR so it also contains the ABI. Python build helper
classes use this variable, so it's important to set it correctly.

Support different settings for target and native.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/classes-recipe/python3-dir.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/python3-dir.bbclass b/meta/classes-recipe/python3-dir.bbclass
index 3eb0dea9b2..2b81b776cb 100644
--- a/meta/classes-recipe/python3-dir.bbclass
+++ b/meta/classes-recipe/python3-dir.bbclass
@@ -5,7 +5,12 @@
 #
 
 PYTHON_BASEVERSION = "3.14"
-PYTHON_ABI = ""
-PYTHON_DIR = "python${PYTHON_BASEVERSION}"
+PYTHON_ABI = "${@bb.utils.contains('DISTRO_FEATURES', 'python_freethreading', 't', '', d)}"
+PYTHON_ABI_NATIVE = "${@bb.utils.contains('DISTRO_FEATURES_NATIVE', 'python_freethreading', 't', '', d)}"
+PYTHON_MAINVERSION = "${PYTHON_BASEVERSION}${PYTHON_ABI}"
+PYTHON_MAINVERSION_NATIVE = "${PYTHON_BASEVERSION}${PYTHON_ABI_NATIVE}"
+PYTHON_DIR = "python${PYTHON_MAINVERSION}"
+PYTHON_DIR_NATIVE = "python${PYTHON_MAINVERSION_NATIVE}"
 PYTHON_PN = "python3"
 PYTHON_SITEPACKAGES_DIR = "${libdir}/${PYTHON_DIR}/site-packages"
+PYTHON_SITEPACKAGES_DIR_NATIVE = "${libdir}/${PYTHON_DIR_NATIVE}/site-packages"
-- 
2.53.0

[RFC][PATCH 02/14] python3: Adapt recipe to freethreading

[Zoltán Böszörményi]

Set the previously added PACKAGECONFIG[freethreading] based on
the new DISTRO_FEATURE=python_freethreading.

Now inherit python3-dir. The PYTHON_MAJMIN private variable was
removed, and its usages were replaced with the proper variables
from python3-dir.bbclass to simplify the settings and for
consistency.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 .../recipes-devtools/python/python3_3.14.3.bb | 61 ++++++++++---------
 1 file changed, 32 insertions(+), 29 deletions(-)

diff --git a/meta/recipes-devtools/python/python3_3.14.3.bb b/meta/recipes-devtools/python/python3_3.14.3.bb
index 7a1ad1e386..59e4412b71 100644
--- a/meta/recipes-devtools/python/python3_3.14.3.bb
+++ b/meta/recipes-devtools/python/python3_3.14.3.bb
@@ -47,19 +47,17 @@ UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar"
 
 CVE_PRODUCT = "python:python python_software_foundation:python cpython"
 
-PYTHON_MAJMIN = "3.14"
-
 S = "${UNPACKDIR}/Python-${PV}"
 
 BBCLASSEXTEND = "native nativesdk"
 
-inherit autotools pkgconfig qemu ptest multilib_header update-alternatives
+inherit autotools pkgconfig qemu python3-dir ptest multilib_header update-alternatives
 
 MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}"
 
 ALTERNATIVE:${PN}-dev = "python3-config"
-ALTERNATIVE_LINK_NAME[python3-config] = "${bindir}/python${PYTHON_MAJMIN}-config"
-ALTERNATIVE_TARGET[python3-config] = "${bindir}/python${PYTHON_MAJMIN}-config-${MULTILIB_SUFFIX}"
+ALTERNATIVE_LINK_NAME[python3-config] = "${bindir}/${PYTHON_DIR}-config"
+ALTERNATIVE_TARGET[python3-config] = "${bindir}/${PYTHON_DIR}-config-${MULTILIB_SUFFIX}"
 
 DEPENDS = "\
     autoconf-archive-native \
@@ -86,7 +84,7 @@ EXTRA_OECONF:append:class-native = " --bindir=${bindir}/${PN}"
 EXTRA_OECONF:append:class-target = " --with-build-python=nativepython3 PLATFORM_TRIPLET=${HOST_ARCH}-${HOST_OS}"
 EXTRA_OECONF:append:class-nativesdk = " --with-build-python=nativepython3"
 
-export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/"
+export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/${PYTHON_DIR}/lib-dynload/"
 
 EXTRANATIVEPATH += "python3-native"
 
@@ -111,7 +109,10 @@ CACHED_CONFIGUREVARS:append:libc-musl = "\
 "
 
 # PGO currently causes builds to not be reproducible so disable by default, see YOCTO #13407
-PACKAGECONFIG ??= "editline gdbm ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
+PACKAGECONFIG ??= "editline gdbm \
+    ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)} \
+    ${@bb.utils.contains('DISTRO_FEATURES', 'python_freethreading', 'freethreading', '', d)} \
+"
 PACKAGECONFIG[freethreading] = "--disable-gil,--enable-gil"
 PACKAGECONFIG[readline] = "--with-readline=readline,,readline,,,editline"
 PACKAGECONFIG[editline] = "--with-readline=editline,,libedit,,,readline"
@@ -178,7 +179,7 @@ do_install:prepend() {
 }
 
 do_install:append:class-target() {
-        oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+        oe_multilib_header ${PYTHON_DIR}/pyconfig.h
 }
 
 do_install:append:class-native() {
@@ -203,20 +204,20 @@ do_install:append:class-native() {
 
         # Nothing should be looking into ${B} for python3-native
         sed -i -e 's:${B}:/build/path/unavailable/:g' \
-                ${D}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile
+                ${D}/${libdir}/${PYTHON_DIR}/config-${PYTHON_MAINVERSION}*/Makefile
 
         # disable the lookup in user's site-packages globally
-        sed -i 's#ENABLE_USER_SITE = None#ENABLE_USER_SITE = False#' ${D}${libdir}/python${PYTHON_MAJMIN}/site.py
+        sed -i 's#ENABLE_USER_SITE = None#ENABLE_USER_SITE = False#' ${D}${libdir}/${PYTHON_DIR}/site.py
 
         # python3-config needs to be in /usr/bin and not in a subdir of it to work properly
         mv ${D}/${bindir}/${PN}/python*config ${D}/${bindir}/
 }
 
 do_install:append() {
-        for c in ${D}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py; do
+        for c in ${D}/${libdir}/${PYTHON_DIR}/_sysconfigdata*.py; do
             python3 ${UNPACKDIR}/reformat_sysconfig.py $c
         done
-        rm -f ${D}${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata*.cpython*
+        rm -f ${D}${libdir}/${PYTHON_DIR}/__pycache__/_sysconfigdata*.cpython*
 
         mkdir -p ${D}${libdir}/python-sysconfigdata
         sysconfigfile=`find ${D} -name _sysconfig*.py`
@@ -224,8 +225,8 @@ do_install:append() {
                 -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \
                 -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
                 -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
-                -e "s,^ 'INCLUDEPY'.*, 'INCLUDEPY': '${STAGING_INCDIR}/python${PYTHON_MAJMIN}'\,,g" \
-                -e "s,^ 'CONFINCLUDEPY'.*, 'CONFINCLUDEPY': '${STAGING_INCDIR}/python${PYTHON_MAJMIN}'\,,g" \
+                -e "s,^ 'INCLUDEPY'.*, 'INCLUDEPY': '${STAGING_INCDIR}/${PYTHON_DIR}'\,,g" \
+                -e "s,^ 'CONFINCLUDEPY'.*, 'CONFINCLUDEPY': '${STAGING_INCDIR}/${PYTHON_DIR}'\,,g" \
                 -e "s,${B},/build/path/unavailable/,g" \
                 $sysconfigfile
         cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py
@@ -241,7 +242,7 @@ do_install:append:class-nativesdk () {
     for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
          sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
     done
-    create_wrapper ${D}${bindir}/python${PYTHON_MAJMIN} TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' PYTHONNOUSERSITE='1'
+    create_wrapper ${D}${bindir}/${PYTHON_DIR} TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' PYTHONNOUSERSITE='1'
 }
 
 do_install_ptest:append:class-target:libc-musl () {
@@ -257,7 +258,7 @@ SYSROOT_PREPROCESS_FUNCS:append:class-nativesdk = " provide_target_config_script
 provide_target_config_script() {
         install -d ${SYSROOT_DESTDIR}${prefix}/python-target-config/
         install ${D}/${bindir}/python3-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
-        install ${D}/${bindir}/python${PYTHON_MAJMIN}-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
+        install ${D}/${bindir}/${PYTHON_DIR}-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
 }
 SYSROOT_DIRS += "${prefix}/python-target-config/"
 
@@ -280,13 +281,13 @@ py_package_preprocess () {
                 -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
                 -e 's:${RECIPE_SYSROOT}::g' \
                 -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
-                ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \
-                ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \
-                ${PKGD}/${bindir}/python${PYTHON_MAJMIN}-config
+                ${PKGD}/${libdir}/${PYTHON_DIR}/config-${PYTHON_MAINVERSION}*/Makefile \
+                ${PKGD}/${libdir}/${PYTHON_DIR}/_sysconfigdata*.py \
+                ${PKGD}/${bindir}/${PYTHON_DIR}-config
 
         # Reformat _sysconfigdata after modifying it so that it remains
         # reproducible
-        for c in ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py; do
+        for c in ${PKGD}/${libdir}/${PYTHON_DIR}/_sysconfigdata*.py; do
             python3 ${UNPACKDIR}/reformat_sysconfig.py $c
         done
 
@@ -301,7 +302,7 @@ py_package_preprocess () {
              -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)"
         cd -
 
-        mv ${PKGD}/${bindir}/python${PYTHON_MAJMIN}-config ${PKGD}/${bindir}/python${PYTHON_MAJMIN}-config-${MULTILIB_SUFFIX}
+        mv ${PKGD}/${bindir}/${PYTHON_DIR}-config ${PKGD}/${bindir}/${PYTHON_DIR}-config-${MULTILIB_SUFFIX}
 
         #Remove the unneeded copy of target sysconfig data
         rm -rf ${PKGD}/${libdir}/python-sysconfigdata
@@ -313,7 +314,7 @@ py_package_preprocess () {
                 -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
                 -e 's:${RECIPE_SYSROOT}::g' \
                 -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
-		${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfig_vars*.json
+		${PKGD}/${libdir}/${PYTHON_DIR}/_sysconfig_vars*.json
 }
 
 # We want bytecode precompiled .py files (.pyc's) by default
@@ -414,7 +415,7 @@ do_create_manifest() {
     cp ${UNPACKDIR}/get_module_deps3.py ${WORKDIR}
     cd ${WORKDIR}
     # This needs to be executed by python-native and NOT by HOST's python
-    nativepython3 create_manifest3.py ${PYTHON_MAJMIN}
+    nativepython3 create_manifest3.py ${PYTHON_MAINVERSION}
     cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json
 }
 
@@ -429,11 +430,12 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX
 
 # For historical reasons PN is empty and provided by python3-modules
 FILES:${PN} = ""
+FILES:${PN}-core += "${bindir}/python${PYTHON_MAINVERSION} ${bindir}/python${PYTHON_BASEVERSION}"
 RPROVIDES:${PN}-modules = "${PN}"
 
-FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
-FILES:${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}"
-FILES:${PN}-tkinter += "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so"
+FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAINVERSION} ${bindir}/pydoc3"
+FILES:${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAINVERSION}"
+FILES:${PN}-tkinter += "${libdir}/${PYTHON_DIR}/lib-dynload/_tkinter.*.so"
 
 # provide python-pyvenv from python3-venv
 RPROVIDES:${PN}-venv += "${MLPREFIX}python3-pyvenv"
@@ -441,13 +443,14 @@ RPROVIDES:${PN}-venv += "${MLPREFIX}python3-pyvenv"
 # package libpython3
 PACKAGES =+ "libpython3 libpython3-staticdev"
 FILES:libpython3 = "${libdir}/libpython*.so.*"
-FILES:libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}-*/libpython${PYTHON_MAJMIN}.a"
+FILES:libpython3-staticdev += "${libdir}/${PYTHON_DIR}/config-${PYTHON_MAINVERSION}-*/lib${PYTHON_DIR}.a"
 INSANE_SKIP:${PN}-dev += "dev-elf"
 INSANE_SKIP:${PN}-ptest = "dev-deps"
 
 # catch all the rest (unsorted)
 PACKAGES += "${PN}-misc"
 RDEPENDS:${PN}-misc += "\
+    bash \
     ${PN}-audio \
     ${PN}-codecs \
     ${PN}-core \
@@ -459,7 +462,7 @@ RDEPENDS:${PN}-misc += "\
 RDEPENDS:${PN}-modules:append:class-target = " ${MLPREFIX}python3-misc"
 RDEPENDS:${PN}-modules:append:class-nativesdk = " ${MLPREFIX}python3-misc"
 RDEPENDS:${PN}-modules:append:class-target = " ${@bb.utils.contains('PACKAGECONFIG', 'gdbm', '${MLPREFIX}python3-gdbm', '', d)}"
-FILES:${PN}-misc = "${libdir}/python${PYTHON_MAJMIN} ${libdir}/python${PYTHON_MAJMIN}/lib-dynload"
+FILES:${PN}-misc = "${libdir}/${PYTHON_DIR} ${libdir}/${PYTHON_DIR}/lib-dynload"
 
 # catch manpage
 PACKAGES += "${PN}-man"
@@ -499,5 +502,5 @@ RDEPENDS:${PN}-tests:append:class-nativesdk = " ${MLPREFIX}bash"
 # Python's tests contain large numbers of files we don't need in the recipe sysroots
 SYSROOT_PREPROCESS_FUNCS += " py3_sysroot_cleanup"
 py3_sysroot_cleanup () {
-	rm -rf ${SYSROOT_DESTDIR}${libdir}/python${PYTHON_MAJMIN}/test
+	rm -rf ${SYSROOT_DESTDIR}${libdir}/${PYTHON_DIR}/test
 }
-- 
2.53.0

[RFC][PATCH 03/14] python3native.bbclass: Fix settings

[Zoltán Böszörményi]

With PYTHON_DIR now containing PYTHON_ABI, don't use the latter.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/classes-recipe/python3native.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/python3native.bbclass b/meta/classes-recipe/python3native.bbclass
index da1283d6b3..755a6c156c 100644
--- a/meta/classes-recipe/python3native.bbclass
+++ b/meta/classes-recipe/python3native.bbclass
@@ -20,8 +20,8 @@ export STAGING_LIBDIR
 # find_package(PythonLibs REQUIRED)
 # which ends up using libs/includes from build host
 # Therefore pre-empt that effort
-export PYTHON_LIBRARY = "${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
-export PYTHON_INCLUDE_DIR = "${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
+export PYTHON_LIBRARY = "${STAGING_LIBDIR}/lib${PYTHON_DIR}.so"
+export PYTHON_INCLUDE_DIR = "${STAGING_INCDIR}/${PYTHON_DIR}"
 
 # suppress host user's site-packages dirs.
 export PYTHONNOUSERSITE = "1"
-- 
2.53.0

[RFC][PATCH 04/14] python_pyo3.bbclass: Pass ABI flag and PYTHON_MAINVERSION

[Zoltán Böszörményi]

Setting the ABI flag correctly is necessary for modules using
python_maturin.bbclass and others based on pyo3.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/classes-recipe/python_pyo3.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/python_pyo3.bbclass b/meta/classes-recipe/python_pyo3.bbclass
index 7f5a00f584..c618d711ac 100644
--- a/meta/classes-recipe/python_pyo3.bbclass
+++ b/meta/classes-recipe/python_pyo3.bbclass
@@ -12,7 +12,8 @@
 inherit cargo python3-dir siteinfo
 
 export PYO3_CROSS = "1"
-export PYO3_CROSS_PYTHON_VERSION = "${PYTHON_BASEVERSION}"
+export PYO3_CROSS_PYTHON_VERSION = "${PYTHON_MAINVERSION}"
+export ABIFLAGS = "${PYTHON_ABI}"
 export PYO3_CROSS_LIB_DIR = "${STAGING_LIBDIR}"
 export CARGO_BUILD_TARGET = "${RUST_HOST_SYS}"
 export RUSTFLAGS
@@ -25,6 +26,7 @@ implementation=CPython
 version=${PYTHON_BASEVERSION}
 shared=true
 abi3=false
+abiflags=${PYTHON_ABI}
 lib_name=${PYTHON_DIR}
 lib_dir=${STAGING_LIBDIR}
 pointer_width=${SITEINFO_BITS}
-- 
2.53.0

[RFC][PATCH 05/14] clang: Use PYTHON_MAINVERSION for python module versioning

[Zoltán Böszörményi]

Use the new complete main version setting instead of just
PYTHON_BASEVERSION that does not include PYTHON_ABI.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-devtools/clang/clang_git.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/clang/clang_git.bb b/meta/recipes-devtools/clang/clang_git.bb
index e10c327a2a..d5f8aa777f 100644
--- a/meta/recipes-devtools/clang/clang_git.bb
+++ b/meta/recipes-devtools/clang/clang_git.bb
@@ -46,7 +46,7 @@ PACKAGECONFIG[lld] = "-DCLANG_DEFAULT_LINKER=lld,,,"
 PACKAGECONFIG[lto] = "-DLLVM_ENABLE_LTO=Full -DLLVM_BINUTILS_INCDIR=${STAGING_INCDIR},,binutils,"
 PACKAGECONFIG[thin-lto] = "-DLLVM_ENABLE_LTO=Thin -DLLVM_BINUTILS_INCDIR=${STAGING_INCDIR},,binutils,"
 PACKAGECONFIG[unwindlib] = "-DCLANG_DEFAULT_UNWINDLIB=libunwind,-DCLANG_DEFAULT_UNWINDLIB=libgcc,,"
-PACKAGECONFIG[libclang-python] = "-DCLANG_PYTHON_BINDINGS_VERSIONS=${PYTHON_BASEVERSION},,"
+PACKAGECONFIG[libclang-python] = "-DCLANG_PYTHON_BINDINGS_VERSIONS=${PYTHON_MAINVERSION},,"
 
 OECMAKE_SOURCEPATH = "${S}/clang"
 
-- 
2.53.0

[RFC][PATCH 06/14] gdb, gdb-cross-canadian: Fix settings for python freethreading

[Zoltán Böszörményi]

PYTHON_DIR now contains PYTHON_ABI, don't use the latter.
Simplify the linked library name to use -l${PYTHON_DIR}.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-devtools/gdb/gdb-cross-canadian.inc | 4 ++--
 meta/recipes-devtools/gdb/gdb_17.1.bb            | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-devtools/gdb/gdb-cross-canadian.inc b/meta/recipes-devtools/gdb/gdb-cross-canadian.inc
index 7b4a7719e4..c4c62d02af 100644
--- a/meta/recipes-devtools/gdb/gdb-cross-canadian.inc
+++ b/meta/recipes-devtools/gdb/gdb-cross-canadian.inc
@@ -22,8 +22,8 @@ do_configure:prepend() {
 cat > ${WORKDIR}/python << EOF
 #! /bin/sh
 case "\$2" in
-        --includes) echo "-I${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}/" ;;
-        --ldflags) echo "-Wl,-rpath-link,${STAGING_LIBDIR}/.. -Wl,-rpath,${libdir}/.. -lpthread -ldl -lutil -lm -lpython${PYTHON_BASEVERSION}${PYTHON_ABI}" ;;
+        --includes) echo "-I${STAGING_INCDIR}/${PYTHON_DIR}/" ;;
+        --ldflags) echo "-Wl,-rpath-link,${STAGING_LIBDIR}/.. -Wl,-rpath,${libdir}/.. -lpthread -ldl -lutil -lm -l${PYTHON_DIR}" ;;
         --exec-prefix) echo "${exec_prefix}" ;;
         *) exit 1 ;;
 esac
diff --git a/meta/recipes-devtools/gdb/gdb_17.1.bb b/meta/recipes-devtools/gdb/gdb_17.1.bb
index 9c6db4ca2c..a5d96f01eb 100644
--- a/meta/recipes-devtools/gdb/gdb_17.1.bb
+++ b/meta/recipes-devtools/gdb/gdb_17.1.bb
@@ -26,8 +26,8 @@ do_configure:prepend() {
 		cat > ${WORKDIR}/python << EOF
 #!/bin/sh
 case "\$2" in
-	--includes) echo "-I${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}/" ;;
-	--ldflags) echo "-Wl,-rpath-link,${STAGING_LIBDIR}/.. -Wl,-rpath,${libdir}/.. -lpthread -ldl -lutil -lm -lpython${PYTHON_BASEVERSION}${PYTHON_ABI}" ;;
+	--includes) echo "-I${STAGING_INCDIR}/${PYTHON_DIR}/" ;;
+	--ldflags) echo "-Wl,-rpath-link,${STAGING_LIBDIR}/.. -Wl,-rpath,${libdir}/.. -lpthread -ldl -lutil -lm -l${PYTHON_DIR}" ;;
 	--exec-prefix) echo "${exec_prefix}" ;;
 	*) exit 1 ;;
 esac
-- 
2.53.0

[RFC][PATCH 07/14] boost: Fix settings for python freethreading

[Zoltán Böszörményi]

PYTHON_DIR now contains PYTHON_ABI, fix the settings accordingly
to still find both python builds.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-support/boost/boost.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-support/boost/boost.inc b/meta/recipes-support/boost/boost.inc
index 64a57ddfb2..c76fd7a0c6 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -174,7 +174,7 @@ do_configure() {
 
 	# If we want Python then we need to tell Boost *exactly* where to find it
 	if ${@bb.utils.contains('BOOST_LIBS', 'python', 'true', 'false', d)}; then
-		echo "using python : ${PYTHON_BASEVERSION} : ${STAGING_DIR_HOST}${bindir}/python3 : ${STAGING_DIR_HOST}${includedir}/${PYTHON_DIR}${PYTHON_ABI} : ${STAGING_DIR_HOST}${libdir}/${PYTHON_DIR} ;" >> ${WORKDIR}/user-config.jam
+		echo "using python : ${PYTHON_BASEVERSION} : ${STAGING_DIR_HOST}${bindir}/python3 : ${STAGING_DIR_HOST}${includedir}/${PYTHON_DIR} : ${STAGING_DIR_HOST}${libdir}/python${PYTHON_BASEVERSION} ;" >> ${WORKDIR}/user-config.jam
 	fi
 
 	if ${@bb.utils.contains('BOOST_LIBS', 'mpi', 'true', 'false', d)}; then
-- 
2.53.0

[RFC][PATCH 08/14] lttng-tools: Simplify setting PYTHON_INCLUDE

[Zoltán Böszörményi]

PYTHON_DIR now contains PYTHON_ABI, use it.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-kernel/lttng/lttng-tools_2.14.1.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/lttng/lttng-tools_2.14.1.bb b/meta/recipes-kernel/lttng/lttng-tools_2.14.1.bb
index 3a3f2cff2c..ef80c53e9c 100644
--- a/meta/recipes-kernel/lttng/lttng-tools_2.14.1.bb
+++ b/meta/recipes-kernel/lttng/lttng-tools_2.14.1.bb
@@ -35,7 +35,7 @@ INSANE_SKIP:${PN}-ptest += "dev-deps"
 
 PYTHON_OPTION = "am_cv_python_pyexecdir='${PYTHON_SITEPACKAGES_DIR}' \
                  am_cv_python_pythondir='${PYTHON_SITEPACKAGES_DIR}' \
-                 PYTHON_INCLUDE='-I${STAGING_INCDIR}/python${PYTHON_BASEVERSION}${PYTHON_ABI}' \
+                 PYTHON_INCLUDE='-I${STAGING_INCDIR}/${PYTHON_DIR}' \
 "
 PACKAGECONFIG ??= "${LTTNGUST} kmod"
 PACKAGECONFIG[python] = "--enable-python-bindings ${PYTHON_OPTION},,python3 swig-native"
-- 
2.53.0

[RFC][PATCH 09/14] lttng-ust: Simplify setting PYTHON_INCLUDE

[Zoltán Böszörményi]

PYTHON_DIR now contains PYTHON_ABI, use it.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-kernel/lttng/lttng-ust_2.14.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/lttng/lttng-ust_2.14.0.bb b/meta/recipes-kernel/lttng/lttng-ust_2.14.0.bb
index 1a15c5b420..5c1c74b7a8 100644
--- a/meta/recipes-kernel/lttng/lttng-ust_2.14.0.bb
+++ b/meta/recipes-kernel/lttng/lttng-ust_2.14.0.bb
@@ -8,7 +8,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=3b04e8f34dbcf08198c6618d05e8fe7b"
 
 PYTHON_OPTION = "am_cv_python_pyexecdir='${PYTHON_SITEPACKAGES_DIR}' \
                  am_cv_python_pythondir='${PYTHON_SITEPACKAGES_DIR}' \
-                 PYTHON_INCLUDE='-I${STAGING_INCDIR}/python${PYTHON_BASEVERSION}${PYTHON_ABI}' \
+                 PYTHON_INCLUDE='-I${STAGING_INCDIR}/${PYTHON_DIR}' \
 "
 
 inherit autotools lib_package manpages python3native pkgconfig
-- 
2.53.0

[RFC][PATCH 10/14] libcap-ng-python: Use PYTHON_DIR for FILES

[Zoltán Böszörményi]

PYTHON_DIR now contains PYTHON_ABI, use it for consistency.

Move the installed site-packages directory to the correct
location.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-support/libcap-ng/libcap-ng-python_0.9.1.bb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-support/libcap-ng/libcap-ng-python_0.9.1.bb b/meta/recipes-support/libcap-ng/libcap-ng-python_0.9.1.bb
index f702056f02..4e01c320fd 100644
--- a/meta/recipes-support/libcap-ng/libcap-ng-python_0.9.1.bb
+++ b/meta/recipes-support/libcap-ng/libcap-ng-python_0.9.1.bb
@@ -12,6 +12,9 @@ EXTRA_OECONF += "--with-python3"
 
 do_install() {
     oe_runmake 'DESTDIR=${D}' install -C ${B}/bindings/python3
+    if [ ! -d ${D}${libdir}/${PYTHON_DIR} ]; then
+        mv ${D}${libdir}/python${PYTHON_BASEVERSION} ${D}${libdir}/${PYTHON_DIR}
+    fi
 }
 
-FILES:${PN} = "${libdir}/python${PYTHON_BASEVERSION}"
+FILES:${PN} = "${libdir}/${PYTHON_DIR}"
-- 
2.53.0

[RFC][PATCH 11/14] hwlatdetect: Use PYTHON_DIR to support python freethreading

[Zoltán Böszörményi]

python${PYTHON_BASEVERSION} is not enough now, PYTHON_DIR is
needed for consistency.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-rt/rt-tests/hwlatdetect_git.bb | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-rt/rt-tests/hwlatdetect_git.bb b/meta/recipes-rt/rt-tests/hwlatdetect_git.bb
index 2dcc80965a..d37909cd87 100644
--- a/meta/recipes-rt/rt-tests/hwlatdetect_git.bb
+++ b/meta/recipes-rt/rt-tests/hwlatdetect_git.bb
@@ -8,7 +8,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
 require rt-tests.inc
 inherit python3-dir
 
-EXTRA_OEMAKE += "PYLIB=${libdir}/python${PYTHON_BASEVERSION}/dist-packages"
+EXTRA_OEMAKE += "PYLIB=${libdir}/${PYTHON_DIR}/dist-packages"
 
 do_compile() {
 	oe_runmake hwlatdetect
@@ -18,9 +18,9 @@ do_install() {
         oe_runmake install_hwlatdetect DESTDIR=${D} SBINDIR=${sbindir} \
 	           MANDIR=${mandir} INCLUDEDIR=${includedir}
 
-        sed -i -e '1s,#!.*python.*,#!${bindir}/python3,' ${D}${libdir}/python${PYTHON_BASEVERSION}/dist-packages/hwlatdetect.py
+        sed -i -e '1s,#!.*python.*,#!${bindir}/python3,' ${D}${libdir}/${PYTHON_DIR}/dist-packages/hwlatdetect.py
 }
 
-FILES:${PN} += "${libdir}/python${PYTHON_BASEVERSION}/dist-packages/hwlatdetect.py"
+FILES:${PN} += "${libdir}/${PYTHON_DIR}/dist-packages/hwlatdetect.py"
 RDEPENDS:${PN} = "python3-core "
 RRECOMMENDS:${PN} = "kernel-module-hwlat-detector"
-- 
2.53.0

[RFC][PATCH 12/14] rpm: Support Python freethreading

[Zoltán Böszörményi]

Added a patch to stop using the stable CPython API (of 3.7!)
as it prevents building rpm against a Python build with
freethreading enabled.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 .../files/0001-rpm-Don-t-use-stable-API.patch | 37 +++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_4.20.1.bb       |  1 +
 2 files changed, 38 insertions(+)
 create mode 100644 meta/recipes-devtools/rpm/files/0001-rpm-Don-t-use-stable-API.patch

diff --git a/meta/recipes-devtools/rpm/files/0001-rpm-Don-t-use-stable-API.patch b/meta/recipes-devtools/rpm/files/0001-rpm-Don-t-use-stable-API.patch
new file mode 100644
index 0000000000..3d82fdab3f
--- /dev/null
+++ b/meta/recipes-devtools/rpm/files/0001-rpm-Don-t-use-stable-API.patch
@@ -0,0 +1,37 @@
+From 2fb202730b49538bbe6d1439d419c18a89eb4c91 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Zolt=C3=A1n=20B=C3=B6sz=C3=B6rm=C3=A9nyi?=
+ <zboszor@gmail.com>
+Date: Wed, 25 Mar 2026 08:20:13 +0100
+Subject: [PATCH] rpm: Don't use stable API
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+With Python freethreading enabled, these explicit errors are
+triggered in the Python headers:
+
+  #error "The limited API is not currently supported in the free-threaded build"
+
+Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
+Upstream-Status: Inappropriate [python freethreading specific]
+---
+ python/CMakeLists.txt | 3 ---
+ 1 file changed, 3 deletions(-)
+
+diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt
+index acc003ad6..3f21205c8 100644
+--- a/python/CMakeLists.txt
++++ b/python/CMakeLists.txt
+@@ -18,9 +18,6 @@ Python3_add_library(_rpm
+ 	spec-py.c spec-py.h
+ )
+ 
+-# Select Python stable ABI
+-target_compile_definitions(_rpm PRIVATE Py_LIMITED_API=0x03070000)
+-
+ target_link_libraries(_rpm PRIVATE librpmio librpm librpmbuild librpmsign)
+ 
+ install(TARGETS _rpm
+-- 
+2.53.0
+
diff --git a/meta/recipes-devtools/rpm/rpm_4.20.1.bb b/meta/recipes-devtools/rpm/rpm_4.20.1.bb
index ba967ec1fa..05be3933e0 100644
--- a/meta/recipes-devtools/rpm/rpm_4.20.1.bb
+++ b/meta/recipes-devtools/rpm/rpm_4.20.1.bb
@@ -39,6 +39,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm;branch=rpm-4.20.x;protoc
            file://0002-rpmio-rpmglob.c-avoid-using-GLOB_BRACE-if-undefined-.patch \
            file://0001-CMakeLists.txt-set-libdir-to-CMAKE_INSTALL_FULL_LIBD.patch \
            file://0001-CMakeLists.txt-Fix-checking-for-CFLAGS.patch \
+           file://0001-rpm-Don-t-use-stable-API.patch \
            "
 
 PE = "1"
-- 
2.53.0

[RFC][PATCH 13/14] xcb-proto: Support Python freethreading

[Zoltán Böszörményi]

xcb-proto installs its python module into the ABI-less
site-packages directory, then in turn libxcb won't find it.

Move the installed site-packages directory to the correct
location.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-graphics/xorg-proto/xcb-proto_1.17.0.bb | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/recipes-graphics/xorg-proto/xcb-proto_1.17.0.bb b/meta/recipes-graphics/xorg-proto/xcb-proto_1.17.0.bb
index 2a78a87e77..dd42dba374 100644
--- a/meta/recipes-graphics/xorg-proto/xcb-proto_1.17.0.bb
+++ b/meta/recipes-graphics/xorg-proto/xcb-proto_1.17.0.bb
@@ -19,6 +19,12 @@ SRC_URI[sha256sum] = "2c1bacd2110f4799f74de6ebb714b94cf6f80fb112316b1219480fd225
 
 inherit autotools pkgconfig python3native
 
+do_install:append () {
+    if [ ! -d ${D}${libdir}/${PYTHON_DIR} ]; then
+        mv ${D}${libdir}/python${PYTHON_BASEVERSION} ${D}${libdir}/${PYTHON_DIR}
+    fi
+}
+
 PACKAGES += "python-xcbgen"
 
 FILES:${PN} = ""
-- 
2.53.0

[RFC][PATCH 14/14] libxcb: Support Python freethreading

[Zoltán Böszörményi]

Added xcb-proto-native as a new build dependency and
inherit python3native.

This fixes the build issues when Python freethreading is enabled.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 meta/recipes-graphics/xorg-lib/libxcb_1.17.0.bb | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-graphics/xorg-lib/libxcb_1.17.0.bb b/meta/recipes-graphics/xorg-lib/libxcb_1.17.0.bb
index fcd8aad303..ac2fcd4cc8 100644
--- a/meta/recipes-graphics/xorg-lib/libxcb_1.17.0.bb
+++ b/meta/recipes-graphics/xorg-lib/libxcb_1.17.0.bb
@@ -16,19 +16,17 @@ SRC_URI[sha256sum] = "599ebf9996710fea71622e6e184f3a8ad5b43d0e5fa8c4e407123c88a5
 
 BBCLASSEXTEND = "native nativesdk"
 
-DEPENDS = "xcb-proto xorgproto libxau libpthread-stubs libxdmcp"
+DEPENDS = "xcb-proto-native xcb-proto xorgproto libxau libpthread-stubs libxdmcp"
 
 PACKAGES_DYNAMIC = "^${PN}-.*"
 
 FILES:${PN} = "${libdir}/libxcb.so.*"
 
-inherit autotools pkgconfig features_check
+inherit autotools pkgconfig python3native features_check
 
 # The libxau and others requires x11 in DISTRO_FEATURES
 REQUIRED_DISTRO_FEATURES = "x11"
 
-export PYTHON = "python3"
-
 do_install:append () {
 	chown root.root ${D}${datadir}/doc/${BPN}/tutorial -R
 }
-- 
2.53.0

Re: [RFC][PATCH 00/14] Support Python freethreading

[Alexander Kanavin]

On Wed, 25 Mar 2026 at 12:31, Zoltán Böszörményi <zboszor@gmail.com> wrote:
>
> This patchset allows turning on Python freethreading by using
> a new DISTRO_FEATURES setting:
>
> DISTRO_FEATURES += "python_freethreading"
>
> and optionally:
>
> DISTRO_FEATURES_NATIVE += "python_freethreading"
>
> The change to python3-dir.bbclass is crucial, as all python module
> build helper classes rely on it.

This is fine as an RFC, but otherwise, given the large amount of
tweaks required, and having to choose between one or the other (which
complicates testing), I would wait and see how other distros provide a
free-threading python. Ideally it should be co-installable with
regular python.

Alex

Re: [RFC][PATCH 00/14] Support Python freethreading

[Böszörményi Zoltán]

2026. 03. 25. 13:58 keltezéssel, Alexander Kanavin írta:
> On Wed, 25 Mar 2026 at 12:31, Zoltán Böszörményi <zboszor@gmail.com> wrote:
>> This patchset allows turning on Python freethreading by using
>> a new DISTRO_FEATURES setting:
>>
>> DISTRO_FEATURES += "python_freethreading"
>>
>> and optionally:
>>
>> DISTRO_FEATURES_NATIVE += "python_freethreading"
>>
>> The change to python3-dir.bbclass is crucial, as all python module
>> build helper classes rely on it.
> This is fine as an RFC, but otherwise, given the large amount of
> tweaks required, and having to choose between one or the other (which
> complicates testing), I would wait and see how other distros provide a
> free-threading python. Ideally it should be co-installable with
> regular python.

As I said, this would double the amount of python module
recipe builds.

See the Red Hat bugzilla ticket I created for a different reason
(old bitbake needs old python, still a parallel installation use case):

https://bugzilla.redhat.com/show_bug.cgi?id=2418374

"We don't have the human power to maintain several separate Python stacks packaged as RPM 
packages in Fedora."

Python 3.14.x is present in Fedora 43, only without freethreading.
I expect they won't do a parallel installation with freethreading, either, due to the same 
reason.
When they switch, it will be all or nothing.

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2235 bytes --]

On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via
lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote:

> This patchset allows turning on Python freethreading by using
> a new DISTRO_FEATURES setting:
>
> DISTRO_FEATURES += "python_freethreading"
>
> and optionally:
>
> DISTRO_FEATURES_NATIVE += "python_freethreading"
>
> The change to python3-dir.bbclass is crucial, as all python module
> build helper classes rely on it.
>
> I also added native variants of some settings because another
> series against meta-openembedded is needed to complete this one.
> In that, the change to the fontforge recipe relies on the native
> setting for setting BUILD_LDFLAGS.
>

Looking at those other patches in the series, they don't seem to
be conditional on the distro feature.

Are those changes valid for a build with and without freethreading ?

That's the type of information, along with the benefits of enabling
this would be information expected in a 0/N introductory patch.

Cheers,

Bruce



>
> All of the changes included here were successfully build tested,
> except for the gdb-cross-canadian change. That one failed, but
> it's included for completeness.
>
> Quite a few of python modules were build tested for native and
> target builds, including python3-pytorch from my meta-python-ai
> with reduced RDEPENDS because e.g. python3-onnx there failed
> to build with freethreading enabled.
>
> All in all, this series at least allows to get things going.
> Though maybe not for Yocto 6.0, as it's quite late.
>
> Best regards,
> Zoltán Böszörményi
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#233874):
> https://lists.openembedded.org/g/openembedded-core/message/233874
> Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 4191 bytes --]

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Böszörményi Zoltán]

2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta:
>
>
> On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via lists.openembedded.org 
> <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org> wrote:
>
>     This patchset allows turning on Python freethreading by using
>     a new DISTRO_FEATURES setting:
>
>     DISTRO_FEATURES += "python_freethreading"
>
>     and optionally:
>
>     DISTRO_FEATURES_NATIVE += "python_freethreading"
>
>     The change to python3-dir.bbclass is crucial, as all python module
>     build helper classes rely on it.
>
>     I also added native variants of some settings because another
>     series against meta-openembedded is needed to complete this one.
>     In that, the change to the fontforge recipe relies on the native
>     setting for setting BUILD_LDFLAGS.
>
>
> Looking at those other patches in the series, they don't seem to
> be conditional on the distro feature.
>
> Are those changes valid for a build with and without freethreading ?

Yes!

The main change is in python3-dir.bbclass, and the recipe tweaks
are just adaptations to the fact that only ${PYTHON_DIR} is needed,
not ${PYTHON_DIR}${PYTHON_ABI} from that point.

I was thinking that the introductory mail stressed this enough.

FWIW, less than 30 recipes was spelling the latter out in
openembedded-core and meta-openembedded combined.

> That's the type of information, along with the benefits of enabling
> this would be information expected in a 0/N introductory patch.
>
> Cheers,
>
> Bruce
>
>
>     All of the changes included here were successfully build tested,
>     except for the gdb-cross-canadian change. That one failed, but
>     it's included for completeness.
>
>     Quite a few of python modules were build tested for native and
>     target builds, including python3-pytorch from my meta-python-ai
>     with reduced RDEPENDS because e.g. python3-onnx there failed
>     to build with freethreading enabled.
>
>     All in all, this series at least allows to get things going.
>     Though maybe not for Yocto 6.0, as it's quite late.
>
>     Best regards,
>     Zoltán Böszörményi
>
>     -=-=-=-=-=-=-=-=-=-=-=-
>     Links: You receive all messages sent to this group.
>     View/Reply Online (#233874):
>     https://lists.openembedded.org/g/openembedded-core/message/233874
>     Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810
>     Group Owner: openembedded-core+owner@lists.openembedded.org
>     <mailto:openembedded-core%2Bowner@lists.openembedded.org>
>     Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>     [bruce.ashfield@gmail.com]
>     -=-=-=-=-=-=-=-=-=-=-=-
>
>
>
> -- 
> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 3719 bytes --]

On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com>
wrote:

> 2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta:
> >
> >
> > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via
> lists.openembedded.org
> > <http://lists.openembedded.org> <zboszor=
> gmail.com@lists.openembedded.org> wrote:
> >
> >     This patchset allows turning on Python freethreading by using
> >     a new DISTRO_FEATURES setting:
> >
> >     DISTRO_FEATURES += "python_freethreading"
> >
> >     and optionally:
> >
> >     DISTRO_FEATURES_NATIVE += "python_freethreading"
> >
> >     The change to python3-dir.bbclass is crucial, as all python module
> >     build helper classes rely on it.
> >
> >     I also added native variants of some settings because another
> >     series against meta-openembedded is needed to complete this one.
> >     In that, the change to the fontforge recipe relies on the native
> >     setting for setting BUILD_LDFLAGS.
> >
> >
> > Looking at those other patches in the series, they don't seem to
> > be conditional on the distro feature.
> >
> > Are those changes valid for a build with and without freethreading ?
>
> Yes!
>
> The main change is in python3-dir.bbclass, and the recipe tweaks
> are just adaptations to the fact that only ${PYTHON_DIR} is needed,
> not ${PYTHON_DIR}${PYTHON_ABI} from that point.
>
> I was thinking that the introductory mail stressed this enough.


Obviously not :)

I didn't get that impression at all from reading the introductory
message or looking at the patches themselves.

You have the overview knowledge of the feature, why it is
useful, how it impacts existing recipes, how the migration would
work, what is the compatibility, etc.

The patch reader doesn't, so that all needs to be clearly
supplied.

Bruce



>
>
> FWIW, less than 30 recipes was spelling the latter out in
> openembedded-core and meta-openembedded combined.
>
> > That's the type of information, along with the benefits of enabling
> > this would be information expected in a 0/N introductory patch.
> >
> > Cheers,
> >
> > Bruce
> >
> >
> >     All of the changes included here were successfully build tested,
> >     except for the gdb-cross-canadian change. That one failed, but
> >     it's included for completeness.
> >
> >     Quite a few of python modules were build tested for native and
> >     target builds, including python3-pytorch from my meta-python-ai
> >     with reduced RDEPENDS because e.g. python3-onnx there failed
> >     to build with freethreading enabled.
> >
> >     All in all, this series at least allows to get things going.
> >     Though maybe not for Yocto 6.0, as it's quite late.
> >
> >     Best regards,
> >     Zoltán Böszörményi
> >
> >     -=-=-=-=-=-=-=-=-=-=-=-
> >     Links: You receive all messages sent to this group.
> >     View/Reply Online (#233874):
> >     https://lists.openembedded.org/g/openembedded-core/message/233874
> >     Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810
> >     Group Owner: openembedded-core+owner@lists.openembedded.org
> >     <mailto:openembedded-core%2Bowner@lists.openembedded.org>
> >     Unsubscribe:
> https://lists.openembedded.org/g/openembedded-core/unsub
> >     [bruce.ashfield@gmail.com]
> >     -=-=-=-=-=-=-=-=-=-=-=-
> >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> >
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 6503 bytes --]

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Böszörményi Zoltán]

2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta:
>
>
> On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote:
>
>     2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta:
>     >
>     >
>     > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via lists.openembedded.org
>     <http://lists.openembedded.org>
>     > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org> wrote:
>     >
>     >     This patchset allows turning on Python freethreading by using
>     >     a new DISTRO_FEATURES setting:
>     >
>     >     DISTRO_FEATURES += "python_freethreading"
>     >
>     >     and optionally:
>     >
>     >     DISTRO_FEATURES_NATIVE += "python_freethreading"
>     >
>     >     The change to python3-dir.bbclass is crucial, as all python module
>     >     build helper classes rely on it.
>     >
>     >     I also added native variants of some settings because another
>     >     series against meta-openembedded is needed to complete this one.
>     >     In that, the change to the fontforge recipe relies on the native
>     >     setting for setting BUILD_LDFLAGS.
>     >
>     >
>     > Looking at those other patches in the series, they don't seem to
>     > be conditional on the distro feature.
>     >
>     > Are those changes valid for a build with and without freethreading ?
>
>     Yes!
>
>     The main change is in python3-dir.bbclass, and the recipe tweaks
>     are just adaptations to the fact that only ${PYTHON_DIR} is needed,
>     not ${PYTHON_DIR}${PYTHON_ABI} from that point.
>
>     I was thinking that the introductory mail stressed this enough.
>
>
> Obviously not :)

Obviously. :)

> I didn't get that impression at all from reading the introductory
> message or looking at the patches themselves.
>
> You have the overview knowledge of the feature, why it is
> useful, how it impacts existing recipes, how the migration would
> work, what is the compatibility, etc.

I think the Python 3.14 release notes has enough details.
See also https://www.phoronix.com/news/Python-3.14

With the two series I sent, only a single recipe is left (cmpi-bindings)
in openembedded-core and meta-openembedded that uses
the PYTHON_ABI setting explicitly.

Migration (of python scripts) should not be a problem.
If it worked with 3.14 as is, it works with freethreading enabled.

Compatibility is poor for a lot of modules, that clearly spell this out
in their pyproject.toml or PKG-INFO:

     "Programming Language :: Python :: Free Threading :: 1 - Unstable"

or something not quite production level support.
It's still experimental for 3rd party modules, despite being officially
supported for the interpreter and built-in modules in 3.14.

This is the reason this was an RFC series.

Anyway, the reason I was experimenting with this was that
with heavily threaded python code, an import storm can cause
a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in
importlib._bootstrap according to the backtrace.

I would like to see if Python 3.14 (with and without freethreading)
fixes such crashes.

> The patch reader doesn't, so that all needs to be clearly
> supplied.

Maybe next time for v2 after I have a complete image
successfully built with freethreading enabled.

>
> Bruce
>
>
>
>     FWIW, less than 30 recipes was spelling the latter out in
>     openembedded-core and meta-openembedded combined.
>
>     > That's the type of information, along with the benefits of enabling
>     > this would be information expected in a 0/N introductory patch.
>     >
>     > Cheers,
>     >
>     > Bruce
>     >
>     >
>     >     All of the changes included here were successfully build tested,
>     >     except for the gdb-cross-canadian change. That one failed, but
>     >     it's included for completeness.
>     >
>     >     Quite a few of python modules were build tested for native and
>     >     target builds, including python3-pytorch from my meta-python-ai
>     >     with reduced RDEPENDS because e.g. python3-onnx there failed
>     >     to build with freethreading enabled.
>     >
>     >     All in all, this series at least allows to get things going.
>     >     Though maybe not for Yocto 6.0, as it's quite late.
>     >
>     >     Best regards,
>     >     Zoltán Böszörményi
>     >
>     >     -=-=-=-=-=-=-=-=-=-=-=-
>     >     Links: You receive all messages sent to this group.
>     >     View/Reply Online (#233874):
>     > https://lists.openembedded.org/g/openembedded-core/message/233874
>     >     Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810
>     >     Group Owner: openembedded-core+owner@lists.openembedded.org
>     <mailto:openembedded-core%2Bowner@lists.openembedded.org>
>     >     <mailto:openembedded-core%2Bowner@lists.openembedded.org
>     <mailto:openembedded-core%252Bowner@lists.openembedded.org>>
>     >     Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>     >     [bruce.ashfield@gmail.com]
>     >     -=-=-=-=-=-=-=-=-=-=-=-
>     >
>     >
>     >
>     > --
>     > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
>     > - "Use the force Harry" - Gandalf, Star Trek II
>     >
>
>
>
> -- 
> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 6395 bytes --]

On Wed, Mar 25, 2026 at 9:51 AM Böszörményi Zoltán <zboszor@gmail.com>
wrote:

> 2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta:
> >
> >
> > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com>
> wrote:
> >
> >     2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta:
> >     >
> >     >
> >     > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via
> lists.openembedded.org
> >     <http://lists.openembedded.org>
> >     > <http://lists.openembedded.org> <zboszor=
> gmail.com@lists.openembedded.org> wrote:
> >     >
> >     >     This patchset allows turning on Python freethreading by using
> >     >     a new DISTRO_FEATURES setting:
> >     >
> >     >     DISTRO_FEATURES += "python_freethreading"
> >     >
> >     >     and optionally:
> >     >
> >     >     DISTRO_FEATURES_NATIVE += "python_freethreading"
> >     >
> >     >     The change to python3-dir.bbclass is crucial, as all python
> module
> >     >     build helper classes rely on it.
> >     >
> >     >     I also added native variants of some settings because another
> >     >     series against meta-openembedded is needed to complete this
> one.
> >     >     In that, the change to the fontforge recipe relies on the
> native
> >     >     setting for setting BUILD_LDFLAGS.
> >     >
> >     >
> >     > Looking at those other patches in the series, they don't seem to
> >     > be conditional on the distro feature.
> >     >
> >     > Are those changes valid for a build with and without freethreading
> ?
> >
> >     Yes!
> >
> >     The main change is in python3-dir.bbclass, and the recipe tweaks
> >     are just adaptations to the fact that only ${PYTHON_DIR} is needed,
> >     not ${PYTHON_DIR}${PYTHON_ABI} from that point.
> >
> >     I was thinking that the introductory mail stressed this enough.
> >
> >
> > Obviously not :)
>
> Obviously. :)
>
> > I didn't get that impression at all from reading the introductory
> > message or looking at the patches themselves.
> >
> > You have the overview knowledge of the feature, why it is
> > useful, how it impacts existing recipes, how the migration would
> > work, what is the compatibility, etc.
>
> I think the Python 3.14 release notes has enough details.
> See also https://www.phoronix.com/news/Python-3.14


The information needs to be distilled into your commit, not left
to the patch reader to go off and read release notes. That's the
point I'm trying to make.


>
>
> With the two series I sent, only a single recipe is left (cmpi-bindings)
> in openembedded-core and meta-openembedded that uses
> the PYTHON_ABI setting explicitly.
>
> Migration (of python scripts) should not be a problem.
> If it worked with 3.14 as is, it works with freethreading enabled.
>
> Compatibility is poor for a lot of modules, that clearly spell this out
> in their pyproject.toml or PKG-INFO:
>
>      "Programming Language :: Python :: Free Threading :: 1 - Unstable"
>
> or something not quite production level support.
> It's still experimental for 3rd party modules, despite being officially
> supported for the interpreter and built-in modules in 3.14.
>
> This is the reason this was an RFC series.
>
> Anyway, the reason I was experimenting with this was that
> with heavily threaded python code, an import storm can cause
> a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in
> importlib._bootstrap according to the backtrace.
>
> I would like to see if Python 3.14 (with and without freethreading)
> fixes such crashes.
>

Definitely worth putting in the RFC as well, the motivation for
the change matters!



>
> > The patch reader doesn't, so that all needs to be clearly
> > supplied.
>
> Maybe next time for v2 after I have a complete image
> successfully built with freethreading enabled.
>

Ack'd

Bruce



>
> >
> > Bruce
> >
> >
> >
> >     FWIW, less than 30 recipes was spelling the latter out in
> >     openembedded-core and meta-openembedded combined.
> >
> >     > That's the type of information, along with the benefits of enabling
> >     > this would be information expected in a 0/N introductory patch.
> >     >
> >     > Cheers,
> >     >
> >     > Bruce
> >     >
> >     >
> >     >     All of the changes included here were successfully build
> tested,
> >     >     except for the gdb-cross-canadian change. That one failed, but
> >     >     it's included for completeness.
> >     >
> >     >     Quite a few of python modules were build tested for native and
> >     >     target builds, including python3-pytorch from my meta-python-ai
> >     >     with reduced RDEPENDS because e.g. python3-onnx there failed
> >     >     to build with freethreading enabled.
> >     >
> >     >     All in all, this series at least allows to get things going.
> >     >     Though maybe not for Yocto 6.0, as it's quite late.
> >     >
> >     >     Best regards,
> >     >     Zoltán Böszörményi
> >     >
> >     >     -=-=-=-=-=-=-=-=-=-=-=-
> >     >     Links: You receive all messages sent to this group.
> >     >     View/Reply Online (#233874):
> >     > https://lists.openembedded.org/g/openembedded-core/message/233874
> >     >     Mute This Topic:
> https://lists.openembedded.org/mt/118499141/1050810
> >     >     Group Owner: openembedded-core+owner@lists.openembedded.org
> >     <mailto:openembedded-core%2Bowner@lists.openembedded.org>
> >     >     <mailto:openembedded-core%2Bowner@lists.openembedded.org
> >     <mailto:openembedded-core%252Bowner@lists.openembedded.org>>
> >     >     Unsubscribe:
> https://lists.openembedded.org/g/openembedded-core/unsub
> >     >     [bruce.ashfield@gmail.com]
> >     >     -=-=-=-=-=-=-=-=-=-=-=-
> >     >
> >     >
> >     >
> >     > --
> >     > - Thou shalt not follow the NULL pointer, for chaos and madness
> await thee at its end
> >     > - "Use the force Harry" - Gandalf, Star Trek II
> >     >
> >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> >
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 10292 bytes --]

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Böszörményi Zoltán]

2026. 03. 25. 15:17 keltezéssel, Bruce Ashfield írta:
>
>
> On Wed, Mar 25, 2026 at 9:51 AM Böszörményi Zoltán <zboszor@gmail.com> wrote:
>
>     2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta:
>     >
>     >
>     > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote:
>     >
>     >     2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta:
>     >     >
>     >     >
>     >     > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via
>     lists.openembedded.org <http://lists.openembedded.org>
>     >     <http://lists.openembedded.org>
>     >     > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org>
>     wrote:
>     >     >
>     >     >     This patchset allows turning on Python freethreading by using
>     >     >     a new DISTRO_FEATURES setting:
>     >     >
>     >     >     DISTRO_FEATURES += "python_freethreading"
>     >     >
>     >     >     and optionally:
>     >     >
>     >     >     DISTRO_FEATURES_NATIVE += "python_freethreading"
>     >     >
>     >     >     The change to python3-dir.bbclass is crucial, as all python module
>     >     >     build helper classes rely on it.
>     >     >
>     >     >     I also added native variants of some settings because another
>     >     >     series against meta-openembedded is needed to complete this one.
>     >     >     In that, the change to the fontforge recipe relies on the native
>     >     >     setting for setting BUILD_LDFLAGS.
>     >     >
>     >     >
>     >     > Looking at those other patches in the series, they don't seem to
>     >     > be conditional on the distro feature.
>     >     >
>     >     > Are those changes valid for a build with and without freethreading ?
>     >
>     >     Yes!
>     >
>     >     The main change is in python3-dir.bbclass, and the recipe tweaks
>     >     are just adaptations to the fact that only ${PYTHON_DIR} is needed,
>     >     not ${PYTHON_DIR}${PYTHON_ABI} from that point.
>     >
>     >     I was thinking that the introductory mail stressed this enough.
>     >
>     >
>     > Obviously not :)
>
>     Obviously. :)
>
>     > I didn't get that impression at all from reading the introductory
>     > message or looking at the patches themselves.
>     >
>     > You have the overview knowledge of the feature, why it is
>     > useful, how it impacts existing recipes, how the migration would
>     > work, what is the compatibility, etc.
>
>     I think the Python 3.14 release notes has enough details.
>     See also https://www.phoronix.com/news/Python-3.14
>
>
> The information needs to be distilled into your commit, not left
> to the patch reader to go off and read release notes. That's the
> point I'm trying to make.
>
>
>
>     With the two series I sent, only a single recipe is left (cmpi-bindings)
>     in openembedded-core and meta-openembedded that uses
>     the PYTHON_ABI setting explicitly.
>
>     Migration (of python scripts) should not be a problem.
>     If it worked with 3.14 as is, it works with freethreading enabled.
>
>     Compatibility is poor for a lot of modules, that clearly spell this out
>     in their pyproject.toml or PKG-INFO:
>
>          "Programming Language :: Python :: Free Threading :: 1 - Unstable"
>
>     or something not quite production level support.
>     It's still experimental for 3rd party modules, despite being officially
>     supported for the interpreter and built-in modules in 3.14.
>
>     This is the reason this was an RFC series.
>
>     Anyway, the reason I was experimenting with this was that
>     with heavily threaded python code, an import storm can cause
>     a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in
>     importlib._bootstrap according to the backtrace.
>
>     I would like to see if Python 3.14 (with and without freethreading)
>     fixes such crashes.
>
>
> Definitely worth putting in the RFC as well, the motivation for
> the change matters!
>
>
>     > The patch reader doesn't, so that all needs to be clearly
>     > supplied.
>
>     Maybe next time for v2 after I have a complete image
>     successfully built with freethreading enabled.
>

No such luck.

lldb in clang won't build because its swig source wants to play games
with PyObject->ob_refcnt directly instead of using the proper macros.
That struct member does not exist when freethreading is enabled.
In fact, the whole struct _object (the PyObject implementation)
is different when freethreading is enabled.

The llvm sources, even in the newly released 22.x and main
do not have any reference to the Py_GIL_DISABLED macro,
so as a whole, it does not support Python freethreading.

I can revisit this series a few year later when llvm adds the
necessary support, which can be backported to older clang
versions in my meta-clang-revival.

This concludes my little experiment.

That being said, I think the cleanup in the recipes is a good direction.
PYTHON_DIR should include PYTHON_ABI in python3-dir.bbclass
because build helper classes rely on it. Then PYTHON_DIR must be
used consistently in recipes that need it, even if PYTHON_ABI stays empty.

>
> Ack'd
>
> Bruce
>
>
>     >
>     > Bruce
>     >
>     >
>     >
>     >     FWIW, less than 30 recipes was spelling the latter out in
>     >     openembedded-core and meta-openembedded combined.
>     >
>     >     > That's the type of information, along with the benefits of enabling
>     >     > this would be information expected in a 0/N introductory patch.
>     >     >
>     >     > Cheers,
>     >     >
>     >     > Bruce
>     >     >
>     >     >
>     >     >     All of the changes included here were successfully build tested,
>     >     >     except for the gdb-cross-canadian change. That one failed, but
>     >     >     it's included for completeness.
>     >     >
>     >     >     Quite a few of python modules were build tested for native and
>     >     >     target builds, including python3-pytorch from my meta-python-ai
>     >     >     with reduced RDEPENDS because e.g. python3-onnx there failed
>     >     >     to build with freethreading enabled.
>     >     >
>     >     >     All in all, this series at least allows to get things going.
>     >     >     Though maybe not for Yocto 6.0, as it's quite late.
>     >     >
>     >     >     Best regards,
>     >     >     Zoltán Böszörményi
>     >     >
>     >     >     -=-=-=-=-=-=-=-=-=-=-=-
>     >     >     Links: You receive all messages sent to this group.
>     >     >     View/Reply Online (#233874):
>     >     > https://lists.openembedded.org/g/openembedded-core/message/233874
>     >     >     Mute This Topic: https://lists.openembedded.org/mt/118499141/1050810
>     >     >     Group Owner: openembedded-core+owner@lists.openembedded.org
>     <mailto:openembedded-core%2Bowner@lists.openembedded.org>
>     >     <mailto:openembedded-core%2Bowner@lists.openembedded.org
>     <mailto:openembedded-core%252Bowner@lists.openembedded.org>>
>     >     >     <mailto:openembedded-core%2Bowner@lists.openembedded.org
>     <mailto:openembedded-core%252Bowner@lists.openembedded.org>
>     >     <mailto:openembedded-core%252Bowner@lists.openembedded.org
>     <mailto:openembedded-core%25252Bowner@lists.openembedded.org>>>
>     >     >     Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>     >     >     [bruce.ashfield@gmail.com]
>     >     >     -=-=-=-=-=-=-=-=-=-=-=-
>     >     >
>     >     >
>     >     >
>     >     > --
>     >     > - Thou shalt not follow the NULL pointer, for chaos and madness await thee
>     at its end
>     >     > - "Use the force Harry" - Gandalf, Star Trek II
>     >     >
>     >
>     >
>     >
>     > --
>     > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
>     > - "Use the force Harry" - Gandalf, Star Trek II
>     >
>
>
>
> -- 
> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>

Re: [OE-core] [RFC][PATCH 00/14] Support Python freethreading

[Böszörményi Zoltán]

2026. 03. 26. 6:29 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> 2026. 03. 25. 15:17 keltezéssel, Bruce Ashfield írta:
>>
>>
>> On Wed, Mar 25, 2026 at 9:51 AM Böszörményi Zoltán <zboszor@gmail.com> wrote:
>>
>>     2026. 03. 25. 14:17 keltezéssel, Bruce Ashfield írta:
>>     >
>>     >
>>     > On Wed, Mar 25, 2026 at 9:13 AM Böszörményi Zoltán <zboszor@gmail.com> wrote:
>>     >
>>     >     2026. 03. 25. 14:01 keltezéssel, Bruce Ashfield írta:
>>     >     >
>>     >     >
>>     >     > On Wed, Mar 25, 2026 at 7:31 AM Zoltan Boszormenyi via
>>     lists.openembedded.org <http://lists.openembedded.org>
>>     >     <http://lists.openembedded.org>
>>     >     > <http://lists.openembedded.org> <zboszor=gmail.com@lists.openembedded.org>
>>     wrote:
>>     >     >
>>     >     >     This patchset allows turning on Python freethreading by using
>>     >     >     a new DISTRO_FEATURES setting:
>>     >     >
>>     >     >     DISTRO_FEATURES += "python_freethreading"
>>     >     >
>>     >     >     and optionally:
>>     >     >
>>     >     >     DISTRO_FEATURES_NATIVE += "python_freethreading"
>>     >     >
>>     >     >     The change to python3-dir.bbclass is crucial, as all python module
>>     >     >     build helper classes rely on it.
>>     >     >
>>     >     >     I also added native variants of some settings because another
>>     >     >     series against meta-openembedded is needed to complete this one.
>>     >     >     In that, the change to the fontforge recipe relies on the native
>>     >     >     setting for setting BUILD_LDFLAGS.
>>     >     >
>>     >     >
>>     >     > Looking at those other patches in the series, they don't seem to
>>     >     > be conditional on the distro feature.
>>     >     >
>>     >     > Are those changes valid for a build with and without freethreading ?
>>     >
>>     >     Yes!
>>     >
>>     >     The main change is in python3-dir.bbclass, and the recipe tweaks
>>     >     are just adaptations to the fact that only ${PYTHON_DIR} is needed,
>>     >     not ${PYTHON_DIR}${PYTHON_ABI} from that point.
>>     >
>>     >     I was thinking that the introductory mail stressed this enough.
>>     >
>>     >
>>     > Obviously not :)
>>
>>     Obviously. :)
>>
>>     > I didn't get that impression at all from reading the introductory
>>     > message or looking at the patches themselves.
>>     >
>>     > You have the overview knowledge of the feature, why it is
>>     > useful, how it impacts existing recipes, how the migration would
>>     > work, what is the compatibility, etc.
>>
>>     I think the Python 3.14 release notes has enough details.
>>     See also https://www.phoronix.com/news/Python-3.14
>>
>>
>> The information needs to be distilled into your commit, not left
>> to the patch reader to go off and read release notes. That's the
>> point I'm trying to make.
>>
>>
>>
>>     With the two series I sent, only a single recipe is left (cmpi-bindings)
>>     in openembedded-core and meta-openembedded that uses
>>     the PYTHON_ABI setting explicitly.
>>
>>     Migration (of python scripts) should not be a problem.
>>     If it worked with 3.14 as is, it works with freethreading enabled.
>>
>>     Compatibility is poor for a lot of modules, that clearly spell this out
>>     in their pyproject.toml or PKG-INFO:
>>
>>          "Programming Language :: Python :: Free Threading :: 1 - Unstable"
>>
>>     or something not quite production level support.
>>     It's still experimental for 3rd party modules, despite being officially
>>     supported for the interpreter and built-in modules in 3.14.
>>
>>     This is the reason this was an RFC series.
>>
>>     Anyway, the reason I was experimenting with this was that
>>     with heavily threaded python code, an import storm can cause
>>     a crash in 3.12.11 or the manually updated 3.13.12 in Yocto 5.3 in
>>     importlib._bootstrap according to the backtrace.
>>
>>     I would like to see if Python 3.14 (with and without freethreading)
>>     fixes such crashes.
>>
>>
>> Definitely worth putting in the RFC as well, the motivation for
>> the change matters!
>>
>>
>>     > The patch reader doesn't, so that all needs to be clearly
>>     > supplied.
>>
>>     Maybe next time for v2 after I have a complete image
>>     successfully built with freethreading enabled.
>>
>
> No such luck.
>
> lldb in clang won't build because its swig source wants to play games
> with PyObject->ob_refcnt directly instead of using the proper macros.
> That struct member does not exist when freethreading is enabled.
> In fact, the whole struct _object (the PyObject implementation)
> is different when freethreading is enabled.
>
> The llvm sources, even in the newly released 22.x and main
> do not have any reference to the Py_GIL_DISABLED macro,
> so as a whole, it does not support Python freethreading.
>
> I can revisit this series a few year later when llvm adds the
> necessary support, which can be backported to older clang
> versions in my meta-clang-revival.
>
> This concludes my little experiment.
>
> That being said, I think the cleanup in the recipes is a good direction.
> PYTHON_DIR should include PYTHON_ABI in python3-dir.bbclass
> because build helper classes rely on it. Then PYTHON_DIR must be
> used consistently in recipes that need it, even if PYTHON_ABI stays empty.

Which means I can re-send the whole series as a cleanup
or preparation series without the DISTRO_FEATURES enablement
part in python3-dir.bbclass.

That one can stay RFC as a standalone patch, separate from
the cleanup series.

I will re-test my build without enabling freethreading
before I would do that.

>
>>
>> Ack'd
>>
>> Bruce
>>
>>
>>     >
>>     > Bruce
>>     >
>>     >
>>     >
>>     >     FWIW, less than 30 recipes was spelling the latter out in
>>     >     openembedded-core and meta-openembedded combined.
>>     >
>>     >     > That's the type of information, along with the benefits of enabling
>>     >     > this would be information expected in a 0/N introductory patch.
>>     >     >
>>     >     > Cheers,
>>     >     >
>>     >     > Bruce
>>     >     >
>>     >     >
>>     >     >     All of the changes included here were successfully build tested,
>>     >     >     except for the gdb-cross-canadian change. That one failed, but
>>     >     >     it's included for completeness.
>>     >     >
>>     >     >     Quite a few of python modules were build tested for native and
>>     >     >     target builds, including python3-pytorch from my meta-python-ai
>>     >     >     with reduced RDEPENDS because e.g. python3-onnx there failed
>>     >     >     to build with freethreading enabled.
>>     >     >
>>     >     >     All in all, this series at least allows to get things going.
>>     >     >     Though maybe not for Yocto 6.0, as it's quite late.
>>     >     >
>>     >     >     Best regards,
>>     >     >     Zoltán Böszörményi
>>     >     >
>>     >     >
>>     >     >
>>     >     >
>>     >     >
>>     >     > --
>>     >     > - Thou shalt not follow the NULL pointer, for chaos and madness await thee
>>     at its end
>>     >     > - "Use the force Harry" - Gandalf, Star Trek II
>>     >     >
>>     >
>>     >
>>     >
>>     > --
>>     > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its 
>> end
>>     > - "Use the force Harry" - Gandalf, Star Trek II
>>     >
>>
>>
>>
>> -- 
>> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#233951): https://lists.openembedded.org/g/openembedded-core/message/233951
> Mute This Topic: https://lists.openembedded.org/mt/118499141/3617728
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [zboszor@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>