[Buildroot] [PATCH 1/2] package/pkg-cargo.mk: add option to vendor crates offline

[Buildroot] [PATCH 1/2] package/pkg-cargo.mk: add option to vendor crates offline

[William Sherrer]

Buildroot packages cargo-packages in a way the allows for offline builds.
However, vendoring the package dependencies requires internet access since
`cargo vendor` defaults to crates.io.

This commit adds the `--respect-source-config` flag to vendor command in
offline builds. This gives the developer the option of adding a custom
.cargo/config.toml in the build directory which points to an offline crates
mirror or registry.

This is particularly useful in airgapped environments where the full build
and/or package development process takes place.

Signed-off-by: William Sherrer <william.sherrer1@gmail.com>
---
 package/pkg-cargo.mk                | 7 +++++++
 support/download/cargo-post-process | 5 ++++-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/package/pkg-cargo.mk b/package/pkg-cargo.mk
index 47a6353f25..8cf8fe49cf 100644
--- a/package/pkg-cargo.mk
+++ b/package/pkg-cargo.mk
@@ -195,6 +195,13 @@ ifneq ($$($(2)_SUBDIR),)
 $(2)_DOWNLOAD_POST_PROCESS_OPTS += -m$$($(2)_SUBDIR)/Cargo.toml
 endif
 
+# If building offline, and the package needs to vendor offline, 
+# use the -r flag in the post_process_opts to use --respect-source-config.
+# This ensures custom registeries or offline crate mirrors are used.
+ifeq ($(ONLINE),n)
+$(2)_DOWNLOAD_POST_PROCESS_OPTS += -r
+endif
+
 # Because we append vendored info, we can't rely on the values being empty
 # once we eventually get into the generic-package infra. So, we duplicate
 # the heuristics here
diff --git a/support/download/cargo-post-process b/support/download/cargo-post-process
index b0e59ad74d..7047e8e881 100755
--- a/support/download/cargo-post-process
+++ b/support/download/cargo-post-process
@@ -11,11 +11,13 @@ if [ "${BR_CARGO_MANIFEST_PATH}" ]; then
 fi
 
 manifest=Cargo.toml
-while getopts "n:o:m:" OPT; do
+respect_source=""
+while getopts "n:o:m:r" OPT; do
     case "${OPT}" in
     o)  output="${OPTARG}";;
     n)  base_name="${OPTARG}";;
     m)  manifest="${OPTARG}";;
+    r)  respect_source="--respect-source-config";;
     :)  error "option '%s' expects a mandatory argument\n" "${OPTARG}";;
     \?) error "unknown option '%s'\n" "${OPTARG}";;
     esac
@@ -36,6 +38,7 @@ mkdir -p .cargo/
 mkdir -p "${CARGO_HOME}"
 flock "${CARGO_HOME}"/.br-lock \
 cargo vendor \
+    "${respect_source}" \
     --manifest-path "${manifest}" \
     --locked VENDOR \
     > .cargo/config.toml
-- 
2.43.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2] docs/manual/adding-packages-cargo.adoc: add instructions for offline vendoring

[William Sherrer]

This commit adds documentation to the cargo package infrastructure for offline vendoring.
It provides an example config for a user for what config file they'll need and where to
install it if working in an offline airgapped environment.

Signed-off-by: William Sherrer <william.sherrer1@gmail.com>
---
 docs/manual/adding-packages-cargo.adoc | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/docs/manual/adding-packages-cargo.adoc b/docs/manual/adding-packages-cargo.adoc
index 4df758537b..f966a1497f 100644
--- a/docs/manual/adding-packages-cargo.adoc
+++ b/docs/manual/adding-packages-cargo.adoc
@@ -54,6 +54,25 @@ It is still possible to define custom build commands or install
 commands (i.e.  with FOO_BUILD_CMDS and FOO_INSTALL_TARGET_CMDS).
 Those will then replace the commands from the cargo infrastructure.
 
+When building custom cargo packages in an offline environment, Buildroot will
+attempt to download your dependencies from crates.io by default. Buildroot builds 
+the package tarball with everything needed to compile the package offline.
+However, to use an offline crates mirror or registry to download dependencies,
+add a +config.toml+ pointing cargo to the offline registry.
+This file should exist in the +output/build/.cargo/+ directory.
+
+Below is an example config using panamax as a mirror.
+
+----
+01: [source.my-mirror]
+02: registry = "http://panamax.internal/crates.io-index"
+03: 
+04: [source.crates-io]
+05: replace-with = "my-mirror"
+----
+
+This config will need to be installed as part of the build process.
+
 ==== +cargo-package+ reference
 
 The main macros for the Cargo package infrastructure are
@@ -96,6 +115,4 @@ the tarball cached in Buildroot's +DL_DIR+, and therefore the hash of
 the package's tarball doesn't only cover the source of the package
 itself, but also covers the sources of the dependencies. Thus, a change
 injected into one of the dependencies will also be discovered by the
-hash check. In addition,  this mechanism allows the build to be
-performed completely offline since cargo will not do any downloads
-during the build. This mechanism is called vendoring the dependencies.
+hash check. This mechanism is called vendoring the dependencies.
-- 
2.43.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot