[PATCH] rust: rustdoc reproducibility issue fix

[PATCH] rust: rustdoc reproducibility issue fix

[sundeep.kokkonda]

From: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>

The PGO (Profile-guided Optimization) collect data about the typical execution of a program and then use this data to inform optimizations such as inlining, machine-code layout, register allocation, etc.
This optimization is by default disabled in rust sources but enabled in Yocto and causing the issue in rustdoc binary. To fix the issue this optimization is set to it's default 'false'.
More about the optimization: https://doc.rust-lang.org/rustc/profile-guided-optimization.html

Reverted commit #3e50e45917 "rust: Split rustdoc into a separate package"

Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 2 --
 meta/recipes-devtools/rust/rust_1.71.1.bb    | 7 +++----
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index 14ccb0b24d..80e830136f 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -16,8 +16,6 @@ import os
 import datetime
 
 exclude_packages = [
-	'rust-rustdoc',
-	'rust-dbg'
 	]
 
 def is_excluded(package):
diff --git a/meta/recipes-devtools/rust/rust_1.71.1.bb b/meta/recipes-devtools/rust/rust_1.71.1.bb
index 3d176e54c1..a0e9316cfd 100644
--- a/meta/recipes-devtools/rust/rust_1.71.1.bb
+++ b/meta/recipes-devtools/rust/rust_1.71.1.bb
@@ -141,13 +141,14 @@ python do_configure() {
     config.add_section("build")
     config.set("build", "submodules", e(False))
     config.set("build", "docs", e(False))
+    config.set("build", "tools", ["rust-demangler",])
 
     rustc = d.expand("${WORKDIR}/rust-snapshot/bin/rustc")
     config.set("build", "rustc", e(rustc))
 
     # Support for the profiler runtime to generate e.g. coverage report,
     # PGO etc.
-    config.set("build", "profiler", e(True))
+    config.set("build", "profiler", e(False))
 
     cargo = d.expand("${WORKDIR}/rust-snapshot/bin/cargo")
     config.set("build", "cargo", e(cargo))
@@ -225,11 +226,9 @@ do_test_compile () {
 
 ALLOW_EMPTY:${PN} = "1"
 
-PACKAGES =+ "${PN}-rustdoc ${PN}-tools-clippy ${PN}-tools-rustfmt"
-FILES:${PN}-rustdoc = "${bindir}/rustdoc"
+PACKAGES =+ "${PN}-tools-clippy ${PN}-tools-rustfmt"
 FILES:${PN}-tools-clippy = "${bindir}/cargo-clippy ${bindir}/clippy-driver"
 FILES:${PN}-tools-rustfmt = "${bindir}/rustfmt"
-RDEPENDS:${PN}-rustdoc = "${PN}"
 RDEPENDS:${PN}-tools-clippy = "${PN}"
 RDEPENDS:${PN}-tools-rustfmt = "${PN}"
 
-- 
2.42.0

Re: [PATCH] rust: rustdoc reproducibility issue fix

[Sundeep KOKKONDA]

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

Testing done with arm, arm64, x86 & x86_64 arch's.
Below combinations are tested.
- Tested the issue by reproducing and fixing it with the patch,
- with oe-selftest,
- Builds b/w different commits,
- with shared sstate b/w the builds.

[-- Attachment #2: Type: text/html, Size: 329 bytes --]

[PATCH] rust: rustdoc reproducibility issue fix

[sundeep.kokkonda]

From: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>

The 'codegen-units' option split the crate into multiple compilation units for parallel compilation. Currently, this split is causing the rustdoc to generate differnt binary between the builds.
To fix this the codegen-units & the lto options are disabled.

More info about options:
https://doc.rust-lang.org/cargo/reference/profiles.html#codegen-units
https://doc.rust-lang.org/rustc/codegen-options/index.html#lto

Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py  |  2 --
 ...ue-fix-cargo-config-for-codegenunits.patch | 26 +++++++++++++++++++
 meta/recipes-devtools/rust/rust-source.inc    |  1 +
 meta/recipes-devtools/rust/rust_1.75.0.bb     |  1 +
 4 files changed, 28 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/rust/files/rustdoc-repro-issue-fix-cargo-config-for-codegenunits.patch

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index 5460f158e5..3d3f30eebc 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -16,8 +16,6 @@ import os
 import datetime
 
 exclude_packages = [
-       'rust-rustdoc',
-       'rust-dbg'
 	]
 
 def is_excluded(package):
diff --git a/meta/recipes-devtools/rust/files/rustdoc-repro-issue-fix-cargo-config-for-codegenunits.patch b/meta/recipes-devtools/rust/files/rustdoc-repro-issue-fix-cargo-config-for-codegenunits.patch
new file mode 100644
index 0000000000..0aab8772eb
--- /dev/null
+++ b/meta/recipes-devtools/rust/files/rustdoc-repro-issue-fix-cargo-config-for-codegenunits.patch
@@ -0,0 +1,26 @@
+rust: rustdoc reproducibility issue fix
+
+rust: rustdoc reproducibility issue fix
+
+The 'codegen-units' option split the crate into multiple compilation units for parallel compilation. Currently, this split is causing the rustdoc to generate differnt binary between the builds.
+To fix this the codegen-units & the lto options are disabled.
+
+More info about options:
+https://doc.rust-lang.org/cargo/reference/profiles.html#codegen-units
+https://doc.rust-lang.org/rustc/codegen-options/index.html#lto
+
+Upstream-Status: Inappropriate [oe specific]
+
+Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
+---
+--- a/.cargo/config.toml
++++ b/.cargo/config.toml
+@@ -3,3 +3,7 @@
+
+ [source.vendored-sources]
+ directory = "vendor"
++
++[profile.release]
++codegen-units = 1
++
+
diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc
index 20ef5e82bc..facf6eb346 100644
--- a/meta/recipes-devtools/rust/rust-source.inc
+++ b/meta/recipes-devtools/rust/rust-source.inc
@@ -13,6 +13,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
             file://0001-Handle-vendored-sources-when-remapping-paths.patch;patchdir=${RUSTSRC} \
             file://repro-issue-fix-with-v175.patch;patchdir=${RUSTSRC} \
             file://0001-cargo-do-not-write-host-information-into-compilation.patch;patchdir=${RUSTSRC} \
+            file://rustdoc-repro-issue-fix-cargo-config-for-codegenunits.patch;patchdir=${RUSTSRC} \
 "
 SRC_URI[rust.sha256sum] = "4526f786d673e4859ff2afa0bab2ba13c918b796519a25c1acce06dba9542340"
 
diff --git a/meta/recipes-devtools/rust/rust_1.75.0.bb b/meta/recipes-devtools/rust/rust_1.75.0.bb
index bb10c852b4..eae1f28bb8 100644
--- a/meta/recipes-devtools/rust/rust_1.75.0.bb
+++ b/meta/recipes-devtools/rust/rust_1.75.0.bb
@@ -136,6 +136,7 @@ python do_configure() {
     config.add_section("rust")
     config.set("rust", "rpath", e(True))
     config.set("rust", "remap-debuginfo", e(True))
+    config.set("rust", "lto", "\"off\"")
     config.set("rust", "channel", e(d.expand("${RUST_CHANNEL}")))
 
     # Whether or not to optimize the compiler and standard library
-- 
2.43.0