[Buildroot] [PATCH] support/download: fix cargo vendoring

* [Buildroot] [PATCH] support/download: fix cargo vendoring
@ 2023-01-09 16:30 Yann E. MORIN
  2023-01-09 21:28 ` Yann E. MORIN
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: Yann E. MORIN @ 2023-01-09 16:30 UTC (permalink / raw)
  To: buildroot; +Cc: James Hilliard, Romain Naour, Simon Richter, Yann E. MORIN

Commit de4cf253752d (package/{rust, rust-bin}: bump to version 1.66.0)
forgot, despite the big comment above the version strings, to confirm
that the vendoring was still working.

Previously, we were adding the vendoring equivalence manually, but in
commit 04154a651729 (support/download/cargo-post-process: cargo output
for vendor config), we switched to using the output of "vargo vendor"
(on stdout) to support caxses were the vendoring equivalence would be
more complex (e.g. when using crates not hosted on crates.io).

With rust until and including 1.65.0, "cargo vendor" would output (for
crates.io crates) the same output as our manual fixups, except it was
preceded by an empty line. So, to avoid recompting all our hashes, we
added a tweak to strip away the leading empty line in 04154a651729.

But rust 1.66.0 includes [0] which changes the output (on stdout) of
"cargo vendor", where the first empty line is no longer emitted.

This means that our tweak for rust 1.65.0 now strips out an important
part of the cargo vendor output, which renders the archives invalid, and
thus generates different archives, which fail to validate against our
hashes.

Fix this by doing what the comment in the post-process helper states,
and just keep the whole output of "cargo vendor", byt just removing the
"tail --lines=+2". Since that comment is no longer meaningful, we drop
it too.

Now, all our 6 cargo-based packages can be vendored again, without
changing our hashes, but most importantly, with valid archives.

Still, we keep the comment above the versions strings, in the hope that
a future bumper will notice and be more careful at validating the
vendoring.

[0] https://github.com/rust-lang/cargo/pull/11273

Fixes:
    http://autobuild.buildroot.org/results/bea/beac7674bbc9fd2f8777b5861f65afee9c485753/  (bat)
    http://autobuild.buildroot.org/results/d1e/d1ec1ebbde115628a4b8b9099544347242a97c1c/  (dust)
    http://autobuild.buildroot.org/results/f96/f968be895be9ca98b314fdd688ef8d3bdf4e5dfb/  (hyerfine)
    http://autobuild.buildroot.org/results/a0c/a0cdb6cc9493f5248d98f98b13da854e12adc2be/  (ripgrep)
    ... and so many others...

Reported-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: Simon Richter <simon.richter@ptwdosimetry.com>
---
 support/download/cargo-post-process | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)

diff --git a/support/download/cargo-post-process b/support/download/cargo-post-process
index 186e9eb69b..21a6be8dbe 100755
--- a/support/download/cargo-post-process
+++ b/support/download/cargo-post-process
@@ -25,25 +25,11 @@ post_process_unpack "${base_name}" "${output}"
 pushd "${base_name}" > /dev/null
 
 # Create the local .cargo/config with vendor info
-#
-# The first line of the output to stdout is empty.
-# So skip it to have the file start with the vendoring
-# configuration (`tail --lines=+2`).
-#
-# NOTE:
-#   There is  a patch for cargo to remove the first empty line:
-#   See: https://github.com/rust-lang/cargo/pull/11273
-#
-#   The patch already landed in +nightly and will end up
-#   in +stable soon.
-#
-# ->  When updating rust/cargo, the call to `tail` must be removed.
-#
 mkdir -p .cargo/
 cargo vendor \
     --manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} \
     --locked VENDOR \
-    | tail --lines=+2 | tee .cargo/config
+    | tee .cargo/config
 
 popd > /dev/null
 
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 6+ messages in thread