Re: [Buildroot] [PATCH 1/1] support/download/cargo-post-process: use cargo output for vendor config

["Yann E. MORIN" <yann.morin.1998@free.fr>]

Simon, All,

On 2022-10-19 18:15 +0200, Simon Richter spake thusly:
> Use the `cargo vendor` output to generate the vendor configuration.
> 
> Fixes the need to patch the configuration if there are
> dependencies that do not come from crates.io.

Could you provide a bit more details on how this works?

For example, we create a config file with:

    [source.crates-io]
    replace-with = "vendored-sources"
    [source.vendored-sources]
    directory = "VENDOR"

What would a similar config be like when using dependencies not coming
from crates.io?

Also, can we have a package that uses non-crates.io dependencies, so
that we have a test-case to ensure we have no regression?

> Signed-off-by: Simon Richter <simon.richter@ptwdosimetry.com>
> ---
>  support/download/cargo-post-process | 11 ++++-------
>  1 file changed, 4 insertions(+), 7 deletions(-)
> 
> diff --git a/support/download/cargo-post-process b/support/download/cargo-post-process
> index a4a4718a2a..7d6ecdb030 100755
> --- a/support/download/cargo-post-process
> +++ b/support/download/cargo-post-process
> @@ -22,17 +22,14 @@ post_process_unpack "${base_name}" "${output}"
>  
>  # Do the Cargo vendoring
>  pushd "${base_name}" > /dev/null
> -cargo vendor --manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} --locked VENDOR
>  
>  # Create the local .cargo/config with vendor info
>  mkdir -p .cargo/
> -cat <<EOF >.cargo/config
> -[source.crates-io]
> -replace-with = "vendored-sources"
> +cargo vendor \
> +	--manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} \
> +	--locked VENDOR 2>&1 \
> +	| sed -ne '1,/To use vendored sources, add/!p' > .cargo/config

This means that vendoring step is then entirely silent, and there is no
progress report.

Since the download during the vendoring can be quite big, that means a
long pause without any feedback to the user that something is going on.

Also, if there is a download issue, there is no way to find what
happened.

Can we keep the vendoring to stdout as it currently is, and re-run it
right after, just to generate the config.toml file? I.e. something like:

    cargo vendor --manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} --locked VENDOR
    cargo vendor --manifest-path ${BR_CARGO_MANIFEST_PATH-Cargo.toml} --locked VENDOR 2>&1 \
    |sed -ne '1,/To use vendored sources, add/!p' > .cargo/config

I've quickly hacked the existing script to run it twice, and it looks
like the second call generates the exact same output, so we should be
covered.

Finally, unrelated to your change: if the package already has a
.cargo/config.toml, then we irremediably override it. Is there something
in that file that would otherwise be required to do the build?

Can we simply append to the file rather than overwrite it, i.e.:
    .... >> .cargo/config.toml

Note that if we want to change that, this should be a different patch,
of course.

Regards,
Yann E. MORIN.

> -[source.vendored-sources]
> -directory = "VENDOR"
> -EOF
>  popd > /dev/null
>  
>  post_process_repack "$(pwd)" "${base_name}" "${output}"
> -- 
> 2.38.0
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot