Re: [Buildroot] [PATCH] support/download: fix cargo vendoring

["Yann E. MORIN" <yann.morin.1998@free.fr>]

All,

On 2023-01-09 17:30 +0100, Yann E. MORIN spake thusly:
> Commit de4cf253752d (package/{rust, rust-bin}: bump to version 1.66.0)
> forgot, despite the big comment above the version strings, to confirm
> that the vendoring was still working.
> 
> Previously, we were adding the vendoring equivalence manually, but in
> commit 04154a651729 (support/download/cargo-post-process: cargo output
> for vendor config), we switched to using the output of "vargo vendor"
> (on stdout) to support caxses were the vendoring equivalence would be
> more complex (e.g. when using crates not hosted on crates.io).
> 
> With rust until and including 1.65.0, "cargo vendor" would output (for
> crates.io crates) the same output as our manual fixups, except it was
> preceded by an empty line. So, to avoid recompting all our hashes, we
> added a tweak to strip away the leading empty line in 04154a651729.
> 
> But rust 1.66.0 includes [0] which changes the output (on stdout) of
> "cargo vendor", where the first empty line is no longer emitted.
> 
> This means that our tweak for rust 1.65.0 now strips out an important
> part of the cargo vendor output, which renders the archives invalid, and
> thus generates different archives, which fail to validate against our
> hashes.
> 
> Fix this by doing what the comment in the post-process helper states,
> and just keep the whole output of "cargo vendor", byt just removing the
> "tail --lines=+2". Since that comment is no longer meaningful, we drop
> it too.
> 
> Now, all our 6 cargo-based packages can be vendored again, without
> changing our hashes, but most importantly, with valid archives.

Of course, I forgot to account for the 5 python packages that have rust
code; they too are fixed with this change.

Regards,
Yann E. MORIN.

> Still, we keep the comment above the versions strings, in the hope that
> a future bumper will notice and be more careful at validating the
> vendoring.
> 
> [0] https://github.com/rust-lang/cargo/pull/11273
> 
> Fixes:
>     http://autobuild.buildroot.org/results/bea/beac7674bbc9fd2f8777b5861f65afee9c485753/  (bat)
>     http://autobuild.buildroot.org/results/d1e/d1ec1ebbde115628a4b8b9099544347242a97c1c/  (dust)
>     http://autobuild.buildroot.org/results/f96/f968be895be9ca98b314fdd688ef8d3bdf4e5dfb/  (hyerfine)
>     http://autobuild.buildroot.org/results/a0c/a0cdb6cc9493f5248d98f98b13da854e12adc2be/  (ripgrep)
>     ... and so many others...
> 
> Reported-by: Romain Naour <romain.naour@gmail.com>
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: James Hilliard <james.hilliard1@gmail.com>
> Cc: Simon Richter <simon.richter@ptwdosimetry.com>
> ---
>  support/download/cargo-post-process | 16 +---------------
>  1 file changed, 1 insertion(+), 15 deletions(-)
> 
> diff --git a/support/download/cargo-post-process b/support/download/cargo-post-process
> index 186e9eb69b..21a6be8dbe 100755
> --- a/support/download/cargo-post-process
> +++ b/support/download/cargo-post-process
> @@ -25,25 +25,11 @@ post_process_unpack "${base_name}" "${output}"
>  pushd "${base_name}" > /dev/null
>  
>  # Create the local .cargo/config with vendor info
> -#
> -# The first line of the output to stdout is empty.
> -# So skip it to have the file start with the vendoring
> -# configuration (`tail --lines=+2`).
> -#
> -# NOTE:
> -#   There is  a patch for cargo to remove the first empty line:
> -#   See: https://github.com/rust-lang/cargo/pull/11273
> -#
> -#   The patch already landed in +nightly and will end up
> -#   in +stable soon.
> -#
> -# ->  When updating rust/cargo, the call to `tail` must be removed.
> -#
>  mkdir -p .cargo/
>  cargo vendor \
>      --manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} \
>      --locked VENDOR \
> -    | tail --lines=+2 | tee .cargo/config
> +    | tee .cargo/config
>  
>  popd > /dev/null
>  
> -- 
> 2.25.1
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot