[PATCH] python3-cython: make generated source file be reproducible

[PATCH] python3-cython: make generated source file be reproducible

[Hongxu Jia]

While python3 module use cython to build library, the generated source file
is not stable at each build and made the generated library not be reproducible

This commit remove un-predictable string from generated source file,
thus the generated library should be reproducible

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 ...h-prefix-from-metadata-in-the-genera.patch | 70 +++++++++++++++++++
 .../python/python3-cython_3.1.3.bb            |  4 ++
 2 files changed, 74 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch

diff --git a/meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch b/meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch
new file mode 100644
index 0000000000..b560e9d020
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-cython/0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch
@@ -0,0 +1,70 @@
+From f33847ca55e508106999471c1d6a2f7371503a1a Mon Sep 17 00:00:00 2001
+From: Hongxu Jia <hongxu.jia@windriver.com>
+Date: Wed, 17 Sep 2025 01:42:08 -0700
+Subject: [PATCH] Remove build path prefix from metadata in the generated output file
+
+The build path may contain tmp dir which is not predictable, it caused
+the generated output file is not stable at each build and made
+the generated library is not reproducible [1] between builds
+
+vim frozenlist/_frozenlist.cpp
+...
+/* BEGIN: Cython Metadata
+{
+    "distutils": {
+        "depends": [],
+        "language": "c++",
+        "name": "frozenlist._frozenlist",
+        "sources": [
+            "/tmp/.tmp-frozenlist-pep517-cfdvygni/src/frozenlist/_frozenlist.pyx"
+        ]
+    },
+    "module_name": "frozenlist._frozenlist"
+}
+END: Cython Metadata */
+...
+
+Remove build path prefix from metadata, after applied this commit,
+vim frozenlist/_frozenlist.cpp
+...
+/* BEGIN: Cython Metadata
+{
+    "distutils": {
+        "depends": [],
+        "language": "c++",
+        "name": "frozenlist._frozenlist",
+        "sources": [
+            "frozenlist/_frozenlist.pyx"
+        ]
+    },
+    "module_name": "frozenlist._frozenlist"
+}
+END: Cython Metadata */
+...
+
+[1] https://reproducible-builds.org/
+
+Upstream-Status: Submitted [https://github.com/cython/cython/pull/7161]
+Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
+---
+ Cython/Compiler/ModuleNode.py | 4 +++-
+ 1 file changed, 3 insertions(+), 1 deletion(-)
+
+diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
+index 6672cb986..ae1625ea2 100644
+--- a/Cython/Compiler/ModuleNode.py
++++ b/Cython/Compiler/ModuleNode.py
+@@ -779,7 +779,9 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
+         code.put_generated_by()
+         if metadata:
+             code.putln("/* BEGIN: Cython Metadata")
+-            code.putln(json.dumps(metadata, indent=4, sort_keys=True))
++            _metadata = json.dumps(metadata, indent=4, sort_keys=True)
++            _metadata = _metadata.replace(os.getcwd()+'/', '')
++            code.putln(_metadata)
+             code.putln("END: Cython Metadata */")
+             code.putln("")
+ 
+-- 
+2.49.0
+
diff --git a/meta/recipes-devtools/python/python3-cython_3.1.3.bb b/meta/recipes-devtools/python/python3-cython_3.1.3.bb
index dcb61a3634..1e0b28b12b 100644
--- a/meta/recipes-devtools/python/python3-cython_3.1.3.bb
+++ b/meta/recipes-devtools/python/python3-cython_3.1.3.bb
@@ -11,6 +11,10 @@ SRC_URI[sha256sum] = "10ee785e42328924b78f75a74f66a813cb956b4a9bc91c44816d089d59
 
 inherit pypi setuptools3 cython
 
+SRC_URI += " \
+    file://0001-Remove-build-path-prefix-from-metadata-in-the-genera.patch \
+"
+
 # No need to depend on self
 DEPENDS:remove = "python3-cython-native"
 
-- 
2.34.1

Re: [OE-core] [PATCH] python3-cython: make generated source file be reproducible

[Gyorgy Sarvari]

On 9/17/25 14:10, hongxu via lists.openembedded.org wrote:
> +diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
> +index 6672cb986..ae1625ea2 100644
> +--- a/Cython/Compiler/ModuleNode.py
> ++++ b/Cython/Compiler/ModuleNode.py
> +@@ -779,7 +779,9 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
> +         code.put_generated_by()
> +         if metadata:
> +             code.putln("/* BEGIN: Cython Metadata")
> +-            code.putln(json.dumps(metadata, indent=4, sort_keys=True))
> ++            _metadata = json.dumps(metadata, indent=4, sort_keys=True)
> ++            _metadata = _metadata.replace(os.getcwd()+'/', '')

I don't think that this is going to work on Windows due to the different
default path separator (of course for Yocto that doesn't matter much,
but I think upstream cares about it)

Maybe os.path.sep would be more portable?

Re: [PATCH] python3-cython: make generated source file be reproducible

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

On Wed, Sep 17, 2025 at 05:26 AM, Gyorgy Sarvari wrote:

> 
> 
>> +diff --git a/Cython/Compiler/ModuleNode.py
>> b/Cython/Compiler/ModuleNode.py
>> +index 6672cb986..ae1625ea2 100644
>> +--- a/Cython/Compiler/ModuleNode.py
>> ++++ b/Cython/Compiler/ModuleNode.py
>> +@@ -779,7 +779,9 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
>> + code.put_generated_by()
>> + if metadata:
>> + code.putln("/* BEGIN: Cython Metadata")
>> +- code.putln(json.dumps(metadata, indent=4, sort_keys=True))
>> ++ _metadata = json.dumps(metadata, indent=4, sort_keys=True)
>> ++ _metadata = _metadata.replace(os.getcwd()+'/', '')
> 
> I don't think that this is going to work on Windows due to the different
> default path separator (of course for Yocto that doesn't matter much,
> but I think upstream cares about it)

OK, I will use hardcode string 'build_path' to replace actual build path prefix,
it should be portable for both of Linux and Windows.

Please drop this review, v2 incomig

//Hongxu

> 
> Maybe os.path.sep would be more portable?

[-- Attachment #2: Type: text/html, Size: 1219 bytes --]