* [PATCH 1/3] cargo.bbclass: Use --frozen flag for cargo operations
2023-08-02 13:16 [PATCH 0/3] Use frozen flag for cargo build frederic.martinsons
@ 2023-08-02 13:16 ` frederic.martinsons
2023-08-02 13:16 ` [PATCH 2/3] cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies frederic.martinsons
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: frederic.martinsons @ 2023-08-02 13:16 UTC (permalink / raw)
To: openembedded-core; +Cc: Frederic Martinsons
From: Frederic Martinsons <frederic.martinsons@gmail.com>
It supersed the --offline flag and guarantee that Cargo.lock
file will not be modified during the build.
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
meta/classes-recipe/cargo.bbclass | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/classes-recipe/cargo.bbclass b/meta/classes-recipe/cargo.bbclass
index 3ef0bbbb44..8c0b92df8d 100644
--- a/meta/classes-recipe/cargo.bbclass
+++ b/meta/classes-recipe/cargo.bbclass
@@ -39,7 +39,12 @@ MANIFEST_PATH ??= "${S}/${CARGO_SRC_DIR}/Cargo.toml"
RUSTFLAGS ??= ""
BUILD_MODE = "${@['--release', ''][d.getVar('DEBUG_BUILD') == '1']}"
-CARGO_BUILD_FLAGS = "-v --offline --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}"
+# --frozen flag will prevent network access (which is required since only
+# the do_fetch step is authorized to access network)
+# and will require an up to date Cargo.lock file.
+# This force the package being built to already ship a Cargo.lock, in the end
+# this is what we want, at least, for reproducibility of the build.
+CARGO_BUILD_FLAGS = "-v --frozen --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}"
# This is based on the content of CARGO_BUILD_FLAGS and generally will need to
# change if CARGO_BUILD_FLAGS changes.
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/3] cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies
2023-08-02 13:16 [PATCH 0/3] Use frozen flag for cargo build frederic.martinsons
2023-08-02 13:16 ` [PATCH 1/3] cargo.bbclass: Use --frozen flag for cargo operations frederic.martinsons
@ 2023-08-02 13:16 ` frederic.martinsons
2023-08-02 13:16 ` [PATCH 3/3] drop rust-hello-world recipe frederic.martinsons
2023-08-02 15:06 ` [PATCH 0/3] Use frozen flag for cargo build Frédéric Martinsons
3 siblings, 0 replies; 6+ messages in thread
From: frederic.martinsons @ 2023-08-02 13:16 UTC (permalink / raw)
To: openembedded-core; +Cc: Frederic Martinsons
From: Frederic Martinsons <frederic.martinsons@gmail.com>
Now we use --frozen, Cargo.lock cannot be modified by cargo build.
These patched git dependencies requires that the git url is removed
from Cargo.lock.
Fixes #15104
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
meta/classes-recipe/cargo_common.bbclass | 41 ++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/meta/classes-recipe/cargo_common.bbclass b/meta/classes-recipe/cargo_common.bbclass
index db54826ddb..b732a1bd95 100644
--- a/meta/classes-recipe/cargo_common.bbclass
+++ b/meta/classes-recipe/cargo_common.bbclass
@@ -117,6 +117,8 @@ cargo_common_do_configure () {
}
python cargo_common_do_patch_paths() {
+ import shutil
+
cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config")
if not os.path.exists(cargo_config):
return
@@ -146,6 +148,45 @@ python cargo_common_do_patch_paths() {
print('\n[patch."%s"]' % k, file=config)
for name in v:
print(name, file=config)
+
+ if not patches:
+ return
+
+ # Cargo.lock file is needed for to be sure that artifacts
+ # downloaded by the fetch steps are those expected by the
+ # project and that the possible patches are correctly applied.
+ # Moreover since we do not want any modification
+ # of this file (for reproducibility purpose), we prevent it by
+ # using --frozen flag (in CARGO_BUILD_FLAGS) and raise a clear error
+ # here is better than letting cargo tell (in case the file is missing)
+ # "Cargo.lock should be modified but --frozen was given"
+
+ manifest_path = d.getVar("MANIFEST_PATH", True)
+ lockfile = os.path.join(os.path.dirname(manifest_path), "Cargo.lock")
+ if not os.path.exists(lockfile):
+ bb.fatal(f"{lockfile} file doesn't exist")
+
+ # There are patched files and so Cargo.lock should be modified but we use
+ # --frozen so let's handle that modifications here.
+ #
+ # Note that a "better" (more elegant ?) would have been to use cargo update for
+ # patched packages:
+ # cargo update --offline -p package_1 -p package_2
+ # But this is not possible since it requires that cargo local git db
+ # to be populated and this is not the case as we fetch git repo ourself.
+
+ lockfile_orig = lockfile + ".orig"
+ if not os.path.exists(lockfile_orig):
+ shutil.copy(lockfile, lockfile_orig)
+
+ newlines = []
+ with open(lockfile_orig, "r") as f:
+ for line in f.readlines():
+ if not line.startswith("source = \"git"):
+ newlines.append(line)
+
+ with open(lockfile, "w") as f:
+ f.writelines(newlines)
}
do_configure[postfuncs] += "cargo_common_do_patch_paths"
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/3] drop rust-hello-world recipe
2023-08-02 13:16 [PATCH 0/3] Use frozen flag for cargo build frederic.martinsons
2023-08-02 13:16 ` [PATCH 1/3] cargo.bbclass: Use --frozen flag for cargo operations frederic.martinsons
2023-08-02 13:16 ` [PATCH 2/3] cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies frederic.martinsons
@ 2023-08-02 13:16 ` frederic.martinsons
2023-08-02 15:06 ` [PATCH 0/3] Use frozen flag for cargo build Frédéric Martinsons
3 siblings, 0 replies; 6+ messages in thread
From: frederic.martinsons @ 2023-08-02 13:16 UTC (permalink / raw)
To: openembedded-core; +Cc: Frederic Martinsons
From: Frederic Martinsons <frederic.martinsons@gmail.com>
we now have more rust/cargo recipes and test that covers
various use cases.
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
---
meta/conf/distro/include/maintainers.inc | 1 -
meta/lib/oeqa/runtime/cases/rust.py | 12 ----------
.../packagegroup-core-tools-testapps.bb | 3 ---
.../rust-hello-world/0001-enable-LTO.patch | 24 -------------------
.../rust-example/rust-hello-world_git.bb | 19 ---------------
5 files changed, 59 deletions(-)
delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch
delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world_git.bb
diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 1c9c5616d8..202e5739a0 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -734,7 +734,6 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <ross.burton@arm.com>"
RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <ross.burton@arm.com>"
RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <Randy.MacLeod@windriver.com>"
RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <Randy.MacLeod@windriver.com>"
-RECIPE_MAINTAINER:pn-rust-hello-world = "Randy MacLeod <Randy.MacLeod@windriver.com>"
RECIPE_MAINTAINER:pn-rust-llvm = "Randy MacLeod <Randy.MacLeod@windriver.com>"
RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <unassigned@yoctoproject.org>"
RECIPE_MAINTAINER:pn-sato-screenshot = "Ross Burton <ross.burton@arm.com>"
diff --git a/meta/lib/oeqa/runtime/cases/rust.py b/meta/lib/oeqa/runtime/cases/rust.py
index c9c60e16fd..9bf0312669 100644
--- a/meta/lib/oeqa/runtime/cases/rust.py
+++ b/meta/lib/oeqa/runtime/cases/rust.py
@@ -48,15 +48,3 @@ class RustCompileTest(OERuntimeTestCase):
status, output = self.target.run('cargo run --manifest-path=/tmp/hello/Cargo.toml')
msg = 'running compiled file failed, output: %s' % output
self.assertEqual(status, 0, msg=msg)
-
-class RustHelloworldTest(OERuntimeTestCase):
- @OETestDepends(['ssh.SSHTest.test_ssh'])
- @OEHasPackage(['rust-hello-world'])
- def test_rusthelloworld(self):
- cmd = "rust-hello-world"
- status, output = self.target.run(cmd)
- msg = 'Exit status was not 0. Output: %s' % output
- self.assertEqual(status, 0, msg=msg)
-
- msg = 'Incorrect output: %s' % output
- self.assertEqual(output, "Hello, world!", msg=msg)
diff --git a/meta/recipes-core/packagegroups/packagegroup-core-tools-testapps.bb b/meta/recipes-core/packagegroups/packagegroup-core-tools-testapps.bb
index e05e329020..1fee1c925d 100644
--- a/meta/recipes-core/packagegroups/packagegroup-core-tools-testapps.bb
+++ b/meta/recipes-core/packagegroups/packagegroup-core-tools-testapps.bb
@@ -25,8 +25,6 @@ GOTOOLS ?= "go-helloworld"
GOTOOLS:powerpc ?= ""
GOTOOLS:riscv32 ?= ""
-RUSTTOOLS ?= "rust-hello-world"
-
GSTEXAMPLES ?= "gst-examples"
GSTEXAMPLES:riscv64 = ""
@@ -59,5 +57,4 @@ RDEPENDS:${PN} = "\
${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', "${X11GLTOOLS}", "", d)} \
${@bb.utils.contains('DISTRO_FEATURES', '3g', "${3GTOOLS}", "", d)} \
${GOTOOLS} \
- ${RUSTTOOLS} \
"
diff --git a/meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch b/meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch
deleted file mode 100644
index f319545ee1..0000000000
--- a/meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch
+++ /dev/null
@@ -1,24 +0,0 @@
-From fa40b874f6470ec11a8fd7b0c9909d0cdd2d6feb Mon Sep 17 00:00:00 2001
-From: Dan Callaghan <dan.callaghan@opengear.com>
-Date: Fri, 5 Feb 2021 08:56:34 +1000
-Subject: [PATCH] enable LTO
-
-Upstream-Status: Pending
----
- Cargo.toml | 3 +++
- 1 file changed, 3 insertions(+)
-
-diff --git a/Cargo.toml b/Cargo.toml
-index 7a2f6c8..cdb6b5d 100644
---- a/Cargo.toml
-+++ b/Cargo.toml
-@@ -3,3 +3,6 @@
- name = "rust-hello-world"
- version = "0.0.1"
- authors = ["Cody P Schafer <dev@codyps.com>"]
-+
-+[profile.release]
-+lto = true
---
-2.28.0
-
diff --git a/meta/recipes-extended/rust-example/rust-hello-world_git.bb b/meta/recipes-extended/rust-example/rust-hello-world_git.bb
deleted file mode 100644
index 1d91109b51..0000000000
--- a/meta/recipes-extended/rust-example/rust-hello-world_git.bb
+++ /dev/null
@@ -1,19 +0,0 @@
-inherit cargo
-
-SRC_URI = "git://github.com/meta-rust/rust-hello-world.git;protocol=https;branch=master"
-SRCREV="e0fa23f1a3cb1eb1407165bd2fc36d2f6e6ad728"
-LIC_FILES_CHKSUM="file://COPYRIGHT;md5=e6b2207ac3740d2d01141c49208c2147"
-
-SRC_URI += "\
- file://0001-enable-LTO.patch \
- "
-
-UPSTREAM_CHECK_COMMITS = "1"
-
-SUMMARY = "Hello World by Cargo for Rust"
-HOMEPAGE = "https://github.com/meta-rust/rust-hello-world"
-LICENSE = "MIT | Apache-2.0"
-
-S = "${WORKDIR}/git"
-
-BBCLASSEXTEND = "native"
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/3] Use frozen flag for cargo build
2023-08-02 13:16 [PATCH 0/3] Use frozen flag for cargo build frederic.martinsons
` (2 preceding siblings ...)
2023-08-02 13:16 ` [PATCH 3/3] drop rust-hello-world recipe frederic.martinsons
@ 2023-08-02 15:06 ` Frédéric Martinsons
3 siblings, 0 replies; 6+ messages in thread
From: Frédéric Martinsons @ 2023-08-02 15:06 UTC (permalink / raw)
To: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]
Le mer. 2 août 2023, 15:16, <frederic.martinsons@gmail.com> a écrit :
> From: Frederic Martinsons <frederic.martinsons@gmail.com>
>
> This patch series force the usage of --frozen instead of --offline during
> cargo build.
> This has the advantage to be sure that Cargo.lock file will not be
> modified.
> Moreover, raise a clear error when Cargo.lock is not present.
> For this to work, we must modify ourself the Cargo.lock before building
> in case there are git repo that have been patched.
>
> The last commit was for removing rust-hello-world, patch to the
> documentation project will follow.
>
> The following changes since commit
> 692e414aed5313ff275b69e93179aa7c559700f3:
>
> ffmpeg: Fix wrong code found with gas/2.41 (2023-08-01 09:49:14 +0100)
>
> are available in the Git repository at:
>
> https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
>
> Frederic Martinsons (3):
> cargo.bbclass: Use --frozen flag for cargo operations
> cargo_common.bbclass: Handle Cargo.lock modifications for git
> dependencies
> drop rust-hello-world recipe
>
> meta/classes-recipe/cargo.bbclass | 7 +++-
> meta/classes-recipe/cargo_common.bbclass | 41 +++++++++++++++++++
> meta/conf/distro/include/maintainers.inc | 1 -
> meta/lib/oeqa/runtime/cases/rust.py | 12 ------
> .../packagegroup-core-tools-testapps.bb | 3 --
> .../rust-hello-world/0001-enable-LTO.patch | 24 -----------
> .../rust-example/rust-hello-world_git.bb | 19 ---------
> 7 files changed, 47 insertions(+), 60 deletions(-)
> delete mode 100644
> meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch
> delete mode 100644 meta/recipes-extended/rust-example/
> rust-hello-world_git.bb
>
> --
> 2.34.1
>
Sorry, I missed the v2 part in header. These are the follow up after
remarks made yesterday (about removal of rust-hello-world and robustness
when modifying Cargo.lock file in place)
>
[-- Attachment #2: Type: text/html, Size: 3106 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread