[Buildroot] [PATCH] utils/add-custom-hashes: add script to manage global patch dir hashes

* [Buildroot] [PATCH] utils/add-custom-hashes: add script to manage global patch dir hashes
@ 2023-11-07 12:44 Peter Korsgaard
  2023-11-07 13:29 ` Peter Korsgaard
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Peter Korsgaard @ 2023-11-07 12:44 UTC (permalink / raw)
  To: buildroot; +Cc: Yann E . MORIN, Thomas De Schampheleire

Add a script to manage the .hash files in the BR2_GLOBAL_PATCH_DIR for
packages using custom versions.

To use it, run in a configured Buildroot directory, E.G.

  make foo_defconfig; ./utils/add-custom-hashes

We support multiple patch directories in BR2_GLOBAL_PATCH_DIR.  If multiple
directories are specified then use the last one as that is likely to be the
most specific one.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
---
 docs/manual/customize-patches.adoc |   2 +
 utils/add-custom-hashes            | 103 +++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
 create mode 100755 utils/add-custom-hashes

diff --git a/docs/manual/customize-patches.adoc b/docs/manual/customize-patches.adoc
index 72e87c3c04..f33bbcdde7 100644
--- a/docs/manual/customize-patches.adoc
+++ b/docs/manual/customize-patches.adoc
@@ -79,3 +79,5 @@ the first file to exist is used to check a package download:
 
 * +<global-patch-dir>/<packagename>/<packageversion>/<packagename>.hash+
 * +<global-patch-dir>/<packagename>/<packagename>.hash+
+
+The +utils/add-custom-hashes+ script can be used to generate these files.
diff --git a/utils/add-custom-hashes b/utils/add-custom-hashes
new file mode 100755
index 0000000000..857ca3dd4c
--- /dev/null
+++ b/utils/add-custom-hashes
@@ -0,0 +1,103 @@
+#!/usr/bin/env bash
+set -e
+
+# Add hash files for packages with custom versions for
+# BR2_DOWNLOAD_FORCE_CHECK_HASHES=y
+#
+# Run in a configured Buildroot directory, E.G.
+# make foo_defconfig; ./utils/add-custom-hashes
+
+# print BR-style message
+# message <info message>
+message() {
+    tput smso 2>/dev/null
+    echo "$*"
+    tput rmso 2>/dev/null
+}
+
+# print error message and exit
+# die <error message>
+die() {
+    echo "Error: $*" >&2
+    exit 1
+}
+
+# get package(s) for download file, if any
+# get_pkgs <json> <file>
+get_pkgs() {
+    jq --arg file "$2" -r \
+       'to_entries[] | select(.value.downloads[0].source == $file) | .key | strings' "$1"
+}
+
+# get download dir for package
+# get_pkg_dl_dir <json> <package>
+get_pkg_dl_dir() {
+    jq --arg pkg "$2" -r '.[$pkg].dl_dir | strings' "$1"
+}
+
+# generate hash file for download file
+# gen_hash <dir> <file>
+gen_hash() {
+    (
+        cd "$1" && printf '# Locally calculated\nsha256  ' && sha256sum "$2"
+    )
+}
+
+command -v jq || die 'Script needs jq'
+
+[ -e .config ] || \
+  die "No .config found, please run this in a configured Buildroot (O=) directory"
+
+message Collecting data
+
+eval "$(make -s VARS='TOPDIR DL_DIR BR_NO_CHECK_HASH_FOR BR2_GLOBAL_PATCH_DIR' QUOTED_VARS=YES printvars)"
+# global patch dir may already have quotes
+BR2_GLOBAL_PATCH_DIR=$(echo "$BR2_GLOBAL_PATCH_DIR" | tr -d '"')
+
+[ -n "$BR2_GLOBAL_PATCH_DIR" ] || die "No BR2_GLOBAL_PATCH_DIR defined, nothing to do"
+[ -n "$BR_NO_CHECK_HASH_FOR" ] || die "No packages without hashes found, nothing to do"
+
+[ -d "$TOPDIR" ] || die "TOPDIR ($TOPDIR) does not look correct"
+[ -d "$DL_DIR" ] || die "DL_DIR ($DL_DIR) does not look correct"
+
+# patch dir may contain multiple dirs, use the last one
+# shellcheck disable=SC2086 # we need the word splitting
+set -- $BR2_GLOBAL_PATCH_DIR
+if [ $# -gt 1 ]; then
+    BR2_GLOBAL_PATCH_DIR="${!#}";
+    message BR2_GLOBAL_PATCH_DIR contains multiple directories, using "$BR2_GLOBAL_PATCH_DIR"
+fi
+
+# patch dir may be relative to TOPDIR
+case "$BR2_GLOBAL_PATCH_DIR" in
+    /*) ;;
+    *) BR2_GLOBAL_PATCH_DIR="$TOPDIR/$BR2_GLOBAL_PATCH_DIR"
+       ;;
+esac
+
+[ -d "$BR2_GLOBAL_PATCH_DIR" ] \
+  || die "BR2_GLOBAL_PATCH_DIR ($BR2_GLOBAL_PATCH_DIR) does not look correct"
+
+trap 'rm -f "$JSON"' EXIT
+JSON=$(mktemp)
+make show-info > "$JSON"
+
+# ensure files have been downloaded, but without checking
+make BR2_DOWNLOAD_FORCE_CHECK_HASHES= source
+
+message Updating hashes
+
+for file in $BR_NO_CHECK_HASH_FOR; do
+    for pkg in $(get_pkgs "$JSON" "$file"); do
+        HASHFILE="$BR2_GLOBAL_PATCH_DIR/$pkg/$pkg.hash"
+        PKG_DL_DIR=$(get_pkg_dl_dir "$JSON" "$pkg")
+        message "Adding hash for $file to $HASHFILE"
+        mkdir -p "${HASHFILE%/*}"
+        gen_hash "$DL_DIR/$PKG_DL_DIR" "$file" > "$HASHFILE"
+    done
+done
+
+message Verifying hashes
+
+make clean
+make BR2_DOWNLOAD_FORCE_CHECK_HASHES=y source
-- 
2.39.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 4+ messages in thread